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ABSTRACT 


The  purpose  of  this  project  was  to  make  an  assessment  of  the  utility  of  the  Hardware  versus  Manpower  (HARDMAN)  III  m 
performing  a  Cost  and  Operational  Effectiveness  Analyses  (COEA).  The  statement  of  work  required  the  eoniraeior  u>  perUirni 
the  following  tasks:  (1)  conduct  an  assessment  of  the  workload  required  to  qualify  an  analyst  and  maintain  his/her  protieieney 
in  applying  the  HARDMAN  HI  including  the  workload  required  for  standardization  and  eonliguration  control  of  the  software. 
(2)  evaluate  the  HARDMAN  Ill’s  usefulness  to  complete  an  accurate  manpower,  personnel  and  training  (MPT)  analysis  in  support 
of  a  COEA  for  TRADOC  agencies;  and  (3)  determine  the  required  hardware  and  software  modilieations  needed  to  improve  the 
HARDMAN  Ill’s  utility. 

CONCLUSIONS 

1.  The  HARDMAN  III  software  was  developed  and  acquired  without  adhering  to  the  following  key  manpower  and  personnel 
integration  (MANPRINT)  concepts:  (1)  performing  a  needs  analysis  of  what  the  ultimate  user  wants  and  needs  to  perform  an 
accurate  MPT  analyses;  (2)  identifying  the  target  analytical  audience;  and  (3)  performing  periodic  MANPRINT  assessments  to 
preclude  people  problems  (e.g.,  user  friendliness)  from  appearing  in  the  software  development.  The  title  "HARDMAN  111" 
software  is  a  misnomer  since  several  of  the  six  modules  do  not  enable  an  MPT  analyst  to  calculate  a  materiel  system's  MPT 
resource  requirements  data  that  previously  could  be  determined  using  the  HARDMAN  Comparative  Methodology  (HCM).  In 
summary,  it  appears  that  there  was  a  lack  of  intended  user  involvement  in  the  design  phase. 

2.  HARDMAN  III  is  difficult  for  the  MPT  analyst  to  create  a  baseline  comparison  system  (BCS)  (a  key  step  in  the  HCM 
methodology).  HARDMAN  III  does  not  have  the  capability  for  an  analyst  to  perform  a  Training  Resource  Requirements  Analysis 
and  produce  the  myriad  of  training  products  for  each  training  course  such  as  annual  student  inputs,  instructor  manpower 
requirements  and  course  costs. 

3.  The  three-day  training  seminar  provides  the  analyst  with  a  familiarization  and  the  basic  rudiments  of  HARDMAN  111  but  dues 
not  qualify  an  individual  to  conduct  a  HARDMAN  III  analysis  using  the  software.  The  HARDMAN  111  training  seminar  was 
not  designed,  developed  nor  conducted  in  accordance  with  the  systems  approach  to  training  (SAT)  as  contained  in  .Militarv 
Standard  (MIL-STD)  1379D. 

4.  The  COEA  is  considered  as  the  vehicle  for  centralized  MPT  analyses  within  TRADOC.  As  an  integrating  document,  the 
COEA  frequently  addresses  new  concepts  requiring  databases  from  non-traditional  sources  (eg.,  contractor  or  other  DOD  service 
maintenance  data)  that  HARDMAN  III  is  unable  to  provide.  However,  databases  can  be  built  using  HARDMAN  111  resources 
permitting.  Additional  COEA  incompatibilities  with  HARDMAN  III  that  were  detected  included:  (a)  HARDMAN  III  How  models 
break  Reliability,  Availability  and  Maintainability  (RAM)  data  into  individual  missions  while  the  Armored  Gun  System  (AGS) 
COEA  uses  annualized  data;  (b)  the  difficulty  by  which  HARDMAN  III  can  accommodate  MPT  requirements  for  several 
alternatives,  in  addition  to  the  base  case,  often  required  of  COEA  study  plans. 

5.  Our  assessment  of  the  HARDMAN  III  software  detected  numerous  nuisances  including:  an  absence  of  technical  speeifieaiions; 
poor  user  friendliness;  and  an  understatement  of  the  database  and  hardware  requirements  to  operate  the  HARDMAN  Ill  software. 
The  HARDMAN  III  software  lacks  compliance  with  DOD-STD-2167A,  "Defense  System  Softw'are  Development."  The 
inconsistencies  are  discussed  in  Section  4.  In  summary,  it  appears  that  there  was  a  lack  of  centralized  database  design  as  well 
as  adequate  reference  documentation. 

6.  Project  A,  used  to  assess  personnel  performance  in  several  of  the  HARDMAN  III  modules,  appears  to  be  obsolete  and  needs 
to  be  updated  since  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  data  reflects  soldier  capabilities  going  I'aek  to  the 
early  1980’s. 

RECOMMENDATIONS 

1 .  Acceptance  or  sponsorship  of  the  HARDMAN  III  software  by  TRADOC  is  not  recommended  because  of  the  following:  (a) 
operating  and  maintaining  the  HARDMAN  III  is  too  costly  in  terms  of  man-hours  per  year  required  for  configuration  control, 
software  standardization,  data  base  updates,  and  additional  computer  hardware  necessary  to  accommodate  future  growili  in  the 
modules;  (b)  the  value  for  TRADOC  school  Directorate  of  Combat  Development  (DCD)  and  Directorate  of  Training  Development 
(DOTD)  MPT  analysts  is  questionable. 

2.  The  Army  Research  Laboratory  (ARL)  should  consider  developing  a  Computer  Based  Training  (CBT)  tutorial  to  train 
prospective  analysts  (outside  TRADOC)  in  HARDMAN  III.  An  improved  tutorial  integrated  into  the  software  modules  w-ould 
be  a  significant  improvement  and  should  replace  the  threc-day  seminar. 

3.  Assuming  improvements  are  made  to  HARDMAN  III,  ARL  should  salvage  the  best  modules,  scrap  T-CON  and  consolidate 
the  various  databases  into  a  single  library  that  would  rectify  many  of  the  inconsistencies  inherent  in  HARDMAN  III  There  sliould 
be  universal  linkages  among  modules  to  facilitate  the  transfer  of  data.  If  consolidations  are  not  made  among  the  modules, 
recommend  improving  the  module  linkages.  ARL  should  develop  personnel  selection  criteria  for  both  military  and  Department 
of  the  Army  civilian  analysts  for  selection  as  HARDMAN  III  analysts. 


SECTION  1.0 

INTRODUCTION 


1.0  INTRODUCTION 

The  HARDMAN  III  Project  was  initiated  by  the  Army  Research  Institute  (ARI) 
to  resolve  analytic  deficiencies  of  previous  methods  (see  Table  1-1)  and  accomplish 
three  objectives:  (1)  influence  system  design  to  improve  accommodation  of  projected 
MPT  constraints;  (2)  evaluate  materiel  systems  in  development  to  determine  MPT 
requirements  for  acceptable  system  performance  and  availability;  and  (3)  enhance  the 
tools  and  techniques  available  to  manpower  and  personnel  integration  (MANPRINT) 
practitioners.  The  HARDMAN  ill  decision  support  system  developed  by  the  ARL, 
Human  Research  and  Engineering  Directorate  (HRED)  is  an  attempt  to  satisfy  the'  need 
for  integrated  analytical  tools  using  a  new  conceptual  base.  The  concept  involves 
defining  MPT  constraints  for  a  developing  system  and  assessing  the  impact  of  these 
constraints  on  human  task  performance  and  overall  system  performance  before  design 
is  begun.  Design  features,  which  include  ease  of  iteration,  personal  computer  (PC)- 
based,  and  built-in  data  libraries,  make  it  a  promising  capability  for  MANPRINT 
analysts. 

1.1  HARDMAN  III  Description.  (The  remainder  of  this  chapter,  including 
figures  and  tables,  is  from:  MANPRINT  -  An  Approach  to  Systems  Integration,  Edited 
by  Dr.  Harold  R.  Booher,  Chapter  12:  MANPRINT  Tools  and  Techniques,  dated 
1990.) 


The  HARDMAN  III  software  consists  of  six  independent  but  mutually 
supporting  modules  (see  Figure  1-1): 

1.1.1  SPARC  (System  Performance  and  RAM  Criterion  Estimation  AidI 
Module.  SPARC  aids  the  user  in  decomposing  a  mission  description  into  functions 
and  subfunctions  and  then  provides  system  performance  criteria  (e.g.,  time,  accuracy) 
at  the  subfunction  level  required  to  meet  mission  criteria.  SPARC  sets  the  minimum 
acceptable  system  performance  requirements  and  specifies  the  reliability,  availability, 
and  maintainability  (RAM)  requirements  for  the  system  at  the  subsystem  level. 
SPARC  enables  hardware  and  software  designers  to  know  what  the  manned  system  will 
have  to  do  in  order  to  be  minimally  acceptable 

1.1.2  M-CON  (Manpower  Constraints  Aid)  Module.  M-CON  provides  an 
estimate  of  the  Army-wide  and  per  system  manpower  that  is  likely  to  be  available  to 
support  a  developing  system.  Use  of  this  module  involves  setting  manpower 
constraints,  evaluating  the  sensitivity  of  the  system  performance  to  these  constraints, 
and  adjusting  as  necessary.  M-CON  addresses  quantitative  manpower  constraints  for 
maximum  crew  sizes  by  system  and  total  operator  and  maintenance  manpower  available 
at  pay  grade,  military  occupational  specialty  (MOS),  and  maintenance  organizational 
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DEC/VAX  mainframe  computers  to  conduct  analyses). 

Source:  Booher,  Harold  R.  and  Glen  M.  Hewitt,  1990  "MANPRINT  Tools  and  Techniques."  In  Harold  R.  Booher,  ed., 
_ MANPRINT:  An  Approach  to  Systems  Integration.  New  York:  Van  Nostrand  Reinhold,  p.  371. _ 
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Source:  See  Table  1  - 


level  of  detail  (e.g.,  direct  support,  general  support).  M-CON  outputs  are  based  on 
manpower  availability  predictions  of  the  MOS  due  to  replacement  of  systems  already 
in  the  force. 


1.1.3  P-CON  (Personnel  Constraints  Aid)  Module.  P-CON  provides  an  estimate 
of  personnel  aptitude  constraints.  Distribution  of  personnel  characteristics  (e.g., 
armed  forces  entrance  test  scores,  education,  language  levels,  and  gender)  is  displayed 
for  operations  and  maintenance  crews  projected  for  future  years.  P-CON  gives  the 
performance  expected  at  the  task  level  based  on  Project  A  database  (see  Table  1-2) 
and  the  MOS  cutoff  score  required  for  criterion  performance.  This  tool  aids  in 
predicting  the  availability  of  soldiers  with  desired  abilities  and  relates  these 
characteristics  to  soldier  performance. 

1.1.4  T-CON  (Training  Constraints  Estimation  Aid)  Module.  T-CON 
(according  to  the  contractor-Applied  Science  Associates)  is  a  tool  used  for  early 
estimation  of  the  training  burden  associated  with  a  new  materiel  system.  T-CON 
employs  a  database  of  training  data  to  estimate  operator  training,  on  a  function-by¬ 
function  basis,  and  maintainer  training,  on  a  subsystem-by-subsystem  basis.  To  use 
T-CON,  the  user  begins  with  a  model  that  defines  functions  and  subsystems.  This 
model  can  then  be  tailored  to  more  closely  fit  the  user’s  intent.  The  training  estimate 
is  developed  by  the  T-CON  software  by  extracting  from  the  database  the  training 
associated  with  real  Army  systems  for  specific  functions  and  subsystems. 

1.1.5  MAN-SEVAL  (Manpower-Based  System  Evaluation  Aid)  Module.  MAN- 
SEVAL  evaluates  the  quantitative  manpower  requirements  at  the  completion  of  the 
system  design  phase  and  prior  to  the  development  of  any  system  prototypes.  MAN- 
SEVAL  identifies  the  jobs  associated  with  each  design  and  the  tasks,  the  number  of 
operators  and  maintainers,  and  the  occupational  specialty  and  skill  level  associated 
with  each  job.  MAN-SEVAL  deals  with  operator  manpower  through  a  combination 
of  modeling  and  workload  analysis.  It  deals  with  maintainer  manpower  by  modeling 
the  relationship  between  maintenance  manpower  and  system  availability.  It  allows  the 
analyst  to  determine  manpower  requirements  and  compare  them  to  manpower 
availability. 

1.1.6  PER-SEVAL  (Personnel-Based  System  Evaluation  Aid)  Module.  PER- 
SEVAL  assists  analysts  in  identifying  the  aptitudes  of  personnel  needed  to  support  a 
particular  design.  It  enables  the  analyst  to  evaluate  the  human  element  of  materiel 
systems  by  identifying  the  level  of  personnel  characteristics  needed  to  meet  design 
system  performance  requirements  with  fixed  amounts  of  training  under  the  specific 
conditions  in  which  the  system  tasks  will  be  performed.  The  components  of  PER- 
SEVAL  according  to  the  Concepts  on  MPT  Estimation  Final  Report  (see  Appendix  B) 
include  the  following:  (1)  performance  shaping  functions  that  predict  performance  as 
a  function  of  personnel  characteristics  and  training;  (2)  stressor  algorithms  that 


degrade  performance  to  reflect  the  presence  of  critical  environmental  stressors;  and 
(3)  operator  and  maintainer  simulation  models  (using  Microsaint  logic)  that  aggregate 
the  performance  estimates  of  individual  tasks  and  produce  estimates  of  system 
performance.  This  tool  is  designed  to  be  used  to  predict  performance  of  all  aptitude 
levels  of  soldiers  operating  or  maintaining  the  system  under  conditions  specified  by 
the  user. 

1.2  Tool  Identification  and  Selection  Factors.  For  systems  integrators  faced  with 
identifying  and/or  evaluating  MANPRINT  issues  in  a  timely  and  cost-effective  manner, 
it  is  important  that  they  have  the  most  appropriate  tools  available.  The  tools  of  the 
highest  quality  are  those  which  have  sufficient  information  to  answer  the  users’ 
questions.  In  addition,  they  need  to  be  accessible,  highly  flexible,  provide  rapid 
response,  and  be  easy  to  use.  In  order  for  the  systems  integrator  to  assess  the  quality 
of  potential  techniques,  the  quality  of  the  data  sources  which  the  tools  use  would  also 
need  to  be  known.  The  criteria  used  to  evaluate  the  cost  effectiveness  of  a 
MANPRINT  tool  such  as  HARDMAN  III  is  as  follows: 

1.2.1  Tool  Existence.  There  may  be  any  number  of  instances  where  a  tool  does  not 
exist  for  the  practitioner’s  question.  Prior  to  the  HCM  and  predecessor  system 
comparability  analyses,  there  essentially  were  no  tools  which  could  assess  the  impact 
of  a  system  design  on  future  MPT  domains.  There  are  currently  few  techniques 
categorized  as  system  safety  tools.  On  the  other  hand,  there  are  a  large  number  of 
useful  operator  workload  assessment  and  prediction  methodologies.  The  primary 
sources  for  determining  tool  existence  include  the  authors  of  the  various  surveys  and 
investigators  at  the  various  government  laboratories  or  contractor  and  academic 
facilities. 

1.2.2  Tool  Accessibility.  Gaining  a  high  degree  of  confidence  that  a  technique  is 
adequately  accessible  requires  consideration  of  the  following:  Is  the  tool  available 
to  the  general  public  or  is  it  company  proprietary  or  other  otherwise  unavailable? 

1.2.3  Quality  of  Content.  This  is  often  the  most  difficult  factor  to  determine. 
First  consider  the  quality  of  the  source  data.  Second,  the  tool  itself  can  be  evaluated 
by  (a)  its  general  intent  from  published  descriptions  of  the  tool  and  (b)  its  ability  to 
provide  the  expected  information  asked  for  by  the  user.  For  example,  if  a  tool  is 
described  as  appropriate  only  for  engineering  and  manufacturing  development,  it 
probably  has  low  content  quality  for  concept  exploration  and  definition  stage  issues. 
Assessments  by  subject  matter  experts  (SMEs)  or  past  users  provide  the  most 
dependable  means  of  determining  content  quality  for  the  new  user. 

1.2.4  Ability  to  Interface  with  Other  Tools.  There  are  generally  two  problems 
to  consider  here.  First,  is  the  information  from  the  tool  designed  to  interface  with 
other  analyses?  The  interaction  with  other  MANPRINT  domains  may  affect  its  ability 
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to  influence  total  system  performance  issues.  Second,  are  there  software  interface 
implications?  Is  the  tool  operable  on  a  computer  system  that  is  generally  available? 
Can  the  technique  communicate  with  other  software  to  be  used  during  the  development 
of  the  system? 

1.2.5  Ease  of  Use.  The  original  HCM  methodology  provides  an  example  of  a  tool 
which  meets  the  first  three  criteria  above  (paragraphs  1.2. 1-1. 2. 3),  and  is  valuable 
even  though  difficult  to  use.  To  assure  wide  application,  the  tools  need  the 
MANPRINT  philosophy  applied  to  them  as  well.  Some  considerations  are: 

•  Easy  use  of  the  information  system 

•  Quick  response 

•  Flexibility  to  model  at  different  levels  of  specificity 

•  Built-in  job  aids  and  variable  help  levels 

•  Reasonable  training  time  for  new  users 

•  Mobility  or  portability  of  tool  hardware/software 

1.2.6  Cost  to  Operate.  Cost  can  be  evaluated  against  the  above  criteria  in 
choosing  among  several  alternatives;  but  in  many  instances  where  only  one  tool  is 
available,  cost  becomes  the  sole  criterion. 

The  above  criteria  are  used  in  the  present  analysis  to  evaluate  the  utility  of  the 
HARDMAN  III  software  modules. 


SECTION  2.0 

TECHNICAL  APPROACH 


2.0  STATEMENT  OF  WORK 

TRADOC  Analysis  Command  (TRAC)  at  Fort  Benjamin  Harrison,  Indiana, 
directed  that  our  work  be  focused  on  the  following: 

(1)  Conduct  a  "mini  MPT  assessment"  of  the  costs  associated  with  operating 
and  maintaining  the  HARDMAN  III  including:  (a)  the  training  workload  required  to 
qualify  and  maintain  proficiency  in  the  application  of  HARDMAN  III:  (b)  any 
manpower  required  for  configuration  control/standardization  of  the  software;  (C)  the 
type  of  qualifications  needed  for  the  eventual  target  audience  desired;  and  (4)  the  best 
training  media  and  materials  needed  to  train  and  maintain  currency  in  the  software. 

(2)  Evaluate  HARDMAN  Ill’s  usefulness  to  complete  an  accurate  MPT 
analysis  in  support  of  TRADOC  analysts  conducting  a  COEA  for  emerging  materiel 
systems. 

(3)  Determine  hardware  and  software  modifications  needed  to  improve  the 
HARDMAN  Ill’s  utility. 

2.1  TECHNICAL  APPROACH 

The  technical  approach  used  to  evaluate  the  utility  of  the  HARDMAN  III 
software  consisted  of  the  following  steps  (see  Figure  2-1):  (1)  review  the  HARDMAN 
III  system  software  and  documentation  for  all  six  modules;  (2)  interview  and  survey 
selected  SMEs  and  users/developers;  (3)  attend  three-day  HARDMAN  III  training 
seminar  to  evaluate  the  adequacy  of  the  training;  (4)  verify  the  HARDMAN  III 
software  algorithms  by  using  MPT  data  from  the  AGS  as  trial  excursions;  (S) 
determine  MPT  costs  associated  with  HARDMAN  III;  (6)  recommend  modifications 
or  enhancements  required  of  the  HARDMAN  III  software;  and  (7)  provide 
recommendations  on  contractor  versus  in-house  centralized  or  decentralized  control 
of  the  HARDMAN  III  software  for  TRADOC  to  apply  to  emerging  U.S.  Army  systems 
in  the  materiel  acquisition  process. 

2.1.1  Research  of  HARDMAN  Publications.  We  reviewed  the  appropriate 
HARDMAN  III  software  documentation  for  each  of  the  six  modules.  The  review 
included  a  literature  search  of  HARDMAN  11  and  Man  Integrated  Systems  Technology 
(MIST)  predecessor  system  documents  such  as  software  user’s  guide,  instructor 
guides,  and  student  study  guides,  etc.  (see  Appendix  B  for  a  complete  list  of 
publications  and  reference  materials  researched). 
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Reference:  HARDMAN  III  Utility  Statement  of  Work 
Dated  29  May  1992 


2.1.2  Attendance  at  HARDMAN  III  Training  Seminar.  An  AEPCO  employee 
attended  the  three-day  training  seminar  that  was  conducted  by  representatives  from 
DRC  and  MAD  from  29  June  through  1  July  1992.  Additional  issues  that  were 
discussed  with  ARL  at  the  training  seminar  included  the  cost  and  licensing  of  the 
HARDMAN  III  software,  differences  between  HARDMAN  III  versus  HARDMAN  11, 
background  of  Project  A  database  and  the  software  documentation.  Questions  were 
asked  about  the  cost  effectiveness  of  the  seminar  versus  the  practicality  of  a  self- 
paced  tutorial  program  (see  Section  5  for  additional  details). 

2.1.3  Administration  of  HARDMAN  111  Surveys.  Selected  SMEs  from  across  the 
Army  were  surveyed.  An  attempt  was  made  to  achieve  a  representative  sample  of 
potential  HARDMAN  III  users,  especially  those  SMEs  who  either  attended  the  training 
seminar  or  were  involved  in  the  evaluation  of  several  "notional  systems"  performed 
under  ARl  sponsorship.  In  several  cases  the  surveys  were  tailored  to  a  specific 
functional  area  (i.e.,  SPARC  to  RAM  developers).  Appendix  D  contains  a  blank 
copy  of  the  HARDMAN  III  survey. 

2.1.4  Evaluation  of  HARDMAN  111  Software.  The  HARDMAN  III  software 
(Version  2.0)  was  evaluated  (see  Section  3  for  evaluation  details)  for  its  compliance 
with  DoD-STD-2168,  "Defense  System  Software  Quality  Program."  User  friendliness 
was  the  paramount  criterion  used  during  the  HARDMAN  III  software  evaluation. 

2.2  DATA  SOURCES  AND  LIMITATIONS 

The  HARDMAN  III  utility  assessment  included  the  following  steps:  (1) 
Discussions  with  SMEs  both  within  and  outside  of  TRADOC;  (2)  Administering 
HARDMAN  III  surveys  (See  Appendix  E  for  a  complete  list  of  HARDMAN  III 
surveyees  and  their  respective  responses);  and  (3)  Attendance  at  the  HARDMAN  III 
training  seminar.  Several  SMEs  declined  to  be  surveyed  because  of  higher  priority 
workload,  personnel  turbulence  and/or  potential  conflict  of  interest. 

2.3  OVERVIEW  OF  PROJECT  A 

Project  A  was  examined  because  soldier  performance  data  and  characteristics 
were  extracted  and  inserted  in  several  of  the  HARDMAN  III  modules.  Project  A  was 
a  comprehensive  long-range  research  and  development  program  which  the  U.S.  Army 
undertook  to  develop  an  improved  personnel  selection  and  classification  system  for 
enlisted  personnel.  The  Army’s  goal  was  to  increase  its  effectiveness  in  matching 
first-tour  enlisted  manpower  requirements  with  available  personnel  resources,  through 
the  use  of  new  and  improved  selection/classification  tests  to  validly  predict  carefully 
developed  measures  of  job  performance.  The  project  sampled  from  the 
675, 000-person  enlisted  personnel  system  of  the  Army  encompassing  several  hundred 
different  military  occupations. 
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2.3.1  Project  A  Backi»round.  This  research  program  began  in  1980  when  ARI 
started  planning  the  extensive  research  effort  that  would  be  needed  to  develop  the 
desired  system.  In  1982,  a  consortium  led  by  the  Human  Resources  Research 
Organization  (HumRRO),  including  the  American  Institutes  for  Research  (AIR)  and 
the  Personnel  Decisions  Research  Institute  (PDRI),  was  selected  by  ARI  to  undertake 
the  project.  The  total  project  utilized  the  services  of  40  to  50  ARI  and  consortium 
researchers  working  collegially  in  a  variety  of  specialties,  such  as  industrial  and 
organizational  psychology,  operations  research,  management  science,  and  computer 
science. 


2.3.2  Project  A  Objectives.  The  specific  objectives  of  Project  A  were  to: 

(1)  Validate  existing  selection  measures  against  both  existing  and 
project-developed  criteria.  The  latter  were  to  include  both  Army  wide  job 
performance  measures  based  on  newly  developed  rating  scales  and  direct  hands-on 
measures  of  MOS-specific  task  performance. 

(2)  Develop  and  validate  new  selection  and  classification  measures. 

(3)  Validate  intermediate  criteria  (e.g.,  performance  in  training)  as  predictors 
of  later  criteria  (e.g.,  job  performance  ratings)  so  that  better  informed  reassignment 
and  promotion  decisions  could  be  made  throughout  a  soldier’s  career. 

(4)  Determine  the  relative  utility  to  the  Army  of  different  performance  levels 
across  an  MOS. 

(5)  Estimate  the  relative  effectiveness  of  alternative  selection  and 
classification  procedures  in  terms  of  their  validity  and  utility  for  making  operational 
selection  and  classification  decisions. 

2.3.3  Project  A  Stages.  The  research  design  for  the  project  incorporated  three 
main  stages  of  data  collection  and  analysis  in  an  iterative  progression  of  development, 
testing,  evaluation,  and  further  development  of  selection/classification  instruments 
(predictors)  and  measures  of  job  performance  (criteria). 

(1)  In  the  first  phase,  data  from  Army  accessions  in  fiscal  years  81  and  82 
were  evaluated  to  explore  the  relationships  between  the  scores  of  applicants  on  the 
ASVAB,  and  the  applicants  subsequent  performance  in  training,  and  their  scores  on 
the  first-tour  Skill  Qualification  Tests  (SQTs). 

(2)  In  the  second  phase,  a  concurrent  validation  design  was  executed  with 
fiscal  year  (FY)  83/84  accessions.  As  part  of  the  preparation  for  the  concurrent 
validation,  a  "preliminary  battery"  of  perceptual,  spatial,  temperament/personality, 
interest,  and  biodata  predictor  measures  was  assembled  and  used  to  test  several 
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thousand  soldiers  as  they  entered  four  MOSs.  The  data  from  this  "preliminary  battery 
sample"  along  with  information  from  a  large-scale  literature  review  and  a  set  of 
structured,  expert  judgments  were  then  used  to  identify  "best  bet"  measures.  These 
"best  bet"  measures  were  developed,  pilot  tested,  and  refined.  The  refined  test 
battery  was  then  field  tested  to  assess  reliabilities,  "fakeability"  (intentionally 
providing  false  replies  to  test  questions),  practice  effects,  and  so  forth.  The  resulting 
predictor  battery,  now  called  the  "Trial  Battery,"  which  includes  computer  - 
administered  perceptual  and  psychomotor  measures,  was  administered  together  with 
a  comprehensive  set  of  job  performance  indices  based  on  job  knowledge  tests, 
hands-on  job  samples,  and  performance  rating  measures  in  the  concurrent  validation. 

(3)  In  the  third  phase  (the  longitudinal  validation),  all  of  the  measures, 
refined  on  the  basis  of  experience  in  field  testing  and  the  concurrent  validation,  were 
administered  in  a  true  predictive  validity  design.  About  50,000  soldiers  across  20 
MOSs  were  included  in  the  FY  86-87  "Experimental  Predictor  Battery"  administration 
and  subsequent  first-tour  measurement.  About  3500  of  these  soldiers  were  estimated 
for  availability  for  second-tour  performance  measurement  in  FY  91. 

(4)  For  both  the  concurrent  and  longitudinal  validations,  the  sample  of  MOSs 
was  specially  selected  as  a  representative  sample  of  the  Army’s  250+  entry-level 
MOSs.  The  selection  was  based  on  an  initial  clustering  of  MOSs  derived  from  rated 
similarities  of  job  content.  These  MOSs  accounted  for  about  45  percent  of  Army 
accessions. 

In  summary,  use  of  Project  A  data  was  an  attempt  to  "piggy  back"  on  the 
Army’s  MOS  testing  program  to  gather  soldier  performance  data  in  the  interest  of  cost 
effectiveness.  Although  the  data  was  biased  toward  lower  category  enlisted  accessions 
experienced  during  that  period  of  time,  it  was  perceived  by  ARI  that  this  performance 
data  was  better  than  no  data  at  all. 
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SECTION  3.0 

MANPOWER,  PERSONNEL,  TRAINING 
AND  SOFTWARE  ISSUES 


3.1  MANPOWER,  PERSONNEL  AND  TRAINING  ISSUES 

The  HARDMAN  III  software  was  developed  and  acquired  in  piecemeal  fashion 
using  three  different  contractors.  ARL,  in  its  quest  to  produce  a  front-end  analytical 
tool,  may  have  overlooked  several  key  elements  of  the  MANPRINT  concept.  For 
example,  the  Deputy  Chief  of  Staff  for  Personnel  Integration  (DCSPI),  U.S.  Total 
Army  Personnel  Command  (USTAPC)  developed  a  flow  chart  or  "horseblanket"  (see 
Appendix  C)  that  illustrates  how  MANPRINT  is  incorporated  into  the  acquisition 
process  for  the  Major  Automated  Information  Systems  Review  Council  (MAISRC). 
This  "horseblanket"  provides  a  quick  reference  of  the  MANPRINT  actions  required 
during  the  respective  acquisition  phases.  Although  the  "horseblanket"  was  not 
available  when  HARDMAN  III  was  developed,  the  key  MANPRINT  processes  inherent 
in  the  "horseblanket"  have  been  espoused  since  the  early  1980’s.  HARDMAN  III  was 
not  considered  a  major  acquisition  in  accordance  with  the  Army  Systems  Acquisition 
Review  Council  (ASARC)  criteria.  However,  several  of  the  crucial  MANPRINT  steps 
such  as  a  needs  analysis  and  identifying  the  target  audience  may  have  surfaced  MPT 
issues  now  being  raised.  For  example,  applying  the  basic  tenets  contained  in  the 
DCSPI  "horseblanket,"  especially  with  reference  to  MANPRINT  fundamentals  such  as 
an  early  target  audience  description,  user  needs  analysis,  MANPRINT  concerns 
reflected  in  the  statement  of  work  to  industry,  and  periodic  independent  MANPRINT 
assessments  at  key  decision  points.  Application  of  the  MANPRINT  concept  early  in 
the  HARDMAN  III  concept  exploration  and  definition  phase  may  have  surfaced  valid 
MPT  concerns  from  a  large  segment  of  the  U.S.  Army  analytical  community— namely 
TRADOC.  The  Concept  Based  Requirements  System  (CBRS)  is  a  structured  process 
used  by  TRADOC  that  utilizes  a  set  of  procedures  to  identify  and  prioritize  Army  war 
fighting  requirements  for  doctrine,  training,  leader  development,  organizations  and 
materiel.  The  process  used  to  produce  the  HARDMAN  III  software  did  not  use  the 
CBRS  to  determine  what  the  ultimate  user  needs. 

3.2  MANPOWER  ISSUES 

3.2.1  Major  Differences  Between  HARDMAN  Ill  versus  HARDMAN  II.  The 
HARDMAN  III  modules  do  not  provide  the  same  capability  as  the  manual  HCM  or 
HARDMAN  II  in  determining  a  materiel  system’s  MPT  requirements  data.  The  MPT 
analyst  is  able  to  establish  a  BCS,  using  substep  6  of  step  1  of  the  generic  HCM,  in 
order  to  project  a  proposed  system’s  MPT  requirements.  An  analyst,  using 
HARDMAN  III,  could  build  a  "scratch  model"  consisting  of  components  already  in 
the  SPARC  module.  However,  this  procedure  is  complex  and  the  reliability  algorithm 
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accuracy  suspect  since  it  is  based  on  average  daily  usage  vice  a  finite  mission  usage. 
HARDMAN  III  does  not  have  the  capability  to  create  a  BCS  (using  hardware 
components  from  other  services  or  industry)  which  is  considered  to  be  a  very  crucial 
step  in  the  HCM.  For  example,  a  recent  task  analysis  performed  on  the  Advanced 
Field  Artillery  System  (AFAS)  at  Fort  Sill,  OK  by  a  contractor  used  predecessor  task 
data  to  predict  the  future  AFAS  tasks  using  the  HARDMAN  III  software.  The 
methodology  used  was  similar  to  an  Early  Comparability  Analysis.  If  the  HARDMAN 
III  had  the  capability  to  construct  a  BCS  consisting  of  tasks  reflecting  new 
technology,  the  proposed  AFAS  task  list  may  have  been  more  realistic  and  useable  for 
further  MANPRINT  analyses.  Similarly,  step  3  of  the  HCM  Training  Analysis  allows 
the  training  analyst  to  establish  BCS  tasks  (substep  4)  and  evaluate  BCS  tasks  (substep 
5).  Most  significant  is  T-CON’s  inability  to  perform  the  key  training  resource 
requirements  analysis.  The  training  analyst,  via  application  of  the  HCM,  is  able  to 
produce  Quasi-Programs  of  Instruction  (POIs),  evaluate  BCS  courses  of  instruction, 
compute  pipeline  student  loads,  and  determine  the  number  of  instructors  and  annual 
course  costs  using  Army  Training  Requirements  and  Resources  System  (ATRRS)  data. 

3.2.2  Manpower  Constraints  versus  Manpower  Requirements.  M-CON  was  not 
designed  to  generate  a  materiel  system’s  theoretical  manpower  requirements.  M-CON 
forces  the  analyst  to  design  new  Army  systems  for  fielding  within  Army  manpower 
ceilings  and/or  budgetary  constraints.  Conversely,  TRADOC  school  combat 
developers  determine  unconstrained  user  manpower  requirements  to  overcome  the 
predecessor’s  deficiency  or  meet  a  new  threat.  Therein  lies  the  dichotomy  of  using 
an  MPT  tool  that  views  manpower  constraints  vice  manpower  requirements  as  the 
primary  sensitivity  criterion. 

3.2.2. 1  It  is  possible  to  eliminate  M-CON  constraints  to  determine  unconstrained 
manpower  requirements.  Step  5  of  the  M-CON  procedure  contains  the  flow  model 
projection  enabling  the  MPT  analyst  to  adjust  the  Army  end  strength.  The 
documentation  states  that  the  impact  of  doing  this  will  be  to  either  increase  or 
decrease  the  number  of  people  required  in  a  particular  MOS.  By  raising  the  end 
strength,  it  is  possible  for  the  MPT  analyst  to  eliminate  the  constraint  factor. 

3. 2. 2. 2  Another  possible  way  to  eliminate  constraints  would  be  to  adjust  manpower 
constraints,  using  Step  6  of  the  M-CON  procedure,  to  compensate  for  differences  in 
the  number  of  requirements  as  determined  by  Manpower  Requirements  Criteria 
(MARC)  versus  the  actual  number  of  positions  authorized  by  the  Army.  Also,  by 
adjusting  for  operating  versus  authorized  manning,  available  manpower  could  be 
increased  to  meet  conceptual  requirements  (see  M-CON  users  guide  and  help  screens). 
In  summary,  M-CON  was  not  designed  to  generate  manpower  requirements.  If  the 
main  purpose  of  M-CON  is  changed  to  generate  requirements  vice  constraints,  the 
documentation  should  reflect  this. 


3.3  PERSONNEL  ISSUES 


3.3.1  Project  A  Database.  Project  A,  used  to  assess  personnel  performance  in 
several  of  the  HARDMAN  III  modules,  may  be  obsolete  since  the  ASVAB  data  reflects 
soldier  capabilities  in  the  1980s.  Using  ten-year  old  personnel  data  to  predict 
performances  over  the  current  time  frame  and  into  the  future  is  misleading.  The 
current  generation  of  accessions  consists  of  a  much  higher  quality  of  soldier  due  to 
several  social  and  economic  factors  in  comparison  to  the  early  1980s.  Also,  the 
assumption  that  performance  characteristics  from  Project  A  degrade  in  a  linear 
relationship  may  not  be  true,  especially  when  multiple  stressors  are  encountered  by 
soldiers  (e.g.,  adverse  climate  and  NBC  conditions). 

3.3.2  Personnel  Selection  Criteria.  It  appears  that  there  should  be  prerequisite 
criteria  established  prior  to  selecting  U.S.  Army  and  other  DoD  MPT  analysts  for 
attendance  at  the  HARDMAN  III  training.  Our  review  of  all  of  the  previous 
attendance  rosters  for  the  three  day  HARDMAN  III  training  courses  revealed  that 
several  attendees  did  not  possess  the  academic  or  the  occupational  credentials  to 
justify  the  HARDMAN  III  training.  Several  attendees  were  last  minute  substitutions 
that  were  merely  "filling  a  seat"  because  the  training  course  had  been  funded  by  their 
particular  Army  agency  who  had  sponsored  the  training.  Several  of  the  HARDMAN 
III  SMEs  surveyed  recommended  that  minimum  prerequisites  be  established  prior  to 
attending  the  training.  These  prerequisites  included:  (1)  Assignment  to  an  MPT 
analytical  position;  (2)  Knowledge  of  personal  computers  including  Disc  Operating 
Systems;  and  (3)  Minimum  of  1  year  of  MANPRINT  experience. 

3.4  TRAINING  ISSUES 

3.4.1  Three-Dav  HARDMAN  III  Training  Seminar.  The  three-day  training 
seminar  merely  provided  the  author  with  a  familiarization  and  the  basic  rudiments  of 
the  HARDMAN  III.  It  was  conducted  using  a  "guided  tour"  approach  whereby  each 
of  the  students  received  an  initial  briefing  on  each  module  and  subsequently  "walked 
through"  the  various  steps  that  are  necessary  to  perform  a  superficial  analysis.  The 
seminar  was  valuable  in  understanding  the  underpinnings  of  the  modules  and  enabled 
the  student  to  observe  the  reactions  and  perceptions  of  other  similar  students.  The 
student  also  has  the  opportunity  to  ask  questions  in  order  to  clarify  any 
misunderstandings.  Several  of  the  SMEs  surveyed  (see  Appendix  E)  were  of  the 
opinion  that  the  training  provided  an  overview  of  HARDMAN  III  but  did  not 
adequately  prepare  an  MPT  analyst  to  conduct  a  HARDMAN  analysis.  There  was  no 
intent  to  conduct  the  training  in  compliance  with  MIL-STD-1379D,  the  Army’s  SAT 
process  or  best  commercial  practices. 


3-3 


3.5  COEA  INCOMPATIBILITIES  WITH  HARDMAN  III 


The  COEA  is  considered  to  be  the  vehicle  for  centralized  MPT  analyses  within 
TRADOC.  MPT  issues  are  an  integral  portion  of  the  COEA,  from  both  the 
operational  effectiveness  and  cost  perspectives  of  the  analyses.  As  an  integrating 
document,  the  COEA  frequently  addresses  new  concepts  requiring  databases  from  non- 
traditional  sources  (e.g.,  contractor  or  other  DoD  service  maintenance  data).  In  the 
case  of  the  AGS,  government  furnished  MPT  data  (limited  by  the  SOW)  from  the  AGS 
COEA  study  was  used  as  a  bench  mark  reference  to  verify  the  algorithms  and  outputs 
from  the  HARDMAN  III  software  modules.  The  following  problems  were  encountered 
with  the  AGS  COEA  data: 

3.5.1  Outputs.  AGS  data  was  provided  in  a  COEA  format  and  we  made 
comparisons  between  the  outputs  of  the  two  methods  (COEA  and  HARDMAN  III) 
revealing  disparities  in  the  results.  This  is  because  SPARC,  MAN-SEVAL,  and  PER- 
SEVAL  utilize  flow  models  that  break  RAM  data  into  individual  missions,  while  the 
AGS  COEA  uses  annualized  data  (Note:  the  SPARC  module  does  very  limited  RAM, 
mostly  maintainability).  Another  drawback  is  the  HARDMAN  III  software’s 
cumbersomeness  in  generating  multiple  alternatives  within  a  single  workfile  causing 
difficulty  in  computing  MPT  requirements  for  several  alternatives  required  by  COEA 
study  plans.  The  MPT  analyst  is  forced  to  download  the  HARDMAN  III  database  and 
reconstruct  a  different  database  populated  with  equipment  representing  the  COEA 
alternatives  (currently  not  reflected  in  the  HARDMAN  III  modules).  It  appears  that 
the  analyst  could  use  the  HARDMAN  III  software  by  computing  each  alternative  as 
a  separate  analysis  but  this  would  be  labor  intensive  vice  using  a  "spreadsheet" 
analytical  approach. 

3.5.2  Replacement  System.  Creation  of  a  replacement  system  not  in  the  established 
library  was  a  problem.  The  system  being  replaced  by  the  AGS  was  the  M551A1 
Sheridan  tank.  Although  sharing  some  similarities  with  the  AGS,  the  MSSlAl  also 
included  other  replacement  systems  such  as  the  Ml  Abrams  and  M3  Bradley.  Creation 
of  a  new  replacement  system  was  not  as  simple  as  it  appeared.  Some  of  the  formulae 
become  distorted  by  using  replacement  systems  that  are  pieced  together.  For  example, 
the  M-CON  replacement  ratio  algorithms  employed  data  from  the  Ml  Abrams  tank 
instead  of  the  MSSlAl  Sheridan  (the  library  database  contains  the  Ml  as  a 
replacement  system  type,  and  not  the  MSSlAl  Sheridan,  which  has  far  fewer  fielded 
systems).  The  results  of  the  formula  used  to  determine  new  system  densities  grew 
distorted,  making  it  appear  that  many  more  operators  would  be  needed  to  field  far 
more  systems.  It  was  possible  to  create  correct  new  system  densities  by  using  the 
phasing  in/out  schedule  utilizing  the  number  of  fielded  systems  broken  down  by  the 
units  that  they  are  assigned  to.  In  this  case,  they  were  units  fielding  Ml  tanks.  By 
assigning  the  number  of  M551A1  systems  arbitrarily  to  Ml  units,  and  doing  the  same 
for  AGS  systems  to  be  fielded,  it  was  possible  to  calculate  true  system  densities.  The 


difficulty  is  that  one  function  had  to  be  utilized  incorrectly  in  order  to  compensate  for 
a  software  deficiency.  Also,  the  main  driver  behind  reducing  the  AGS  operator  crew 
size  from  four  to  three  was  the  introduction  of  an  ammunition  auto  loader  system, 
which  is  not  a  task  included  in  ihe  HARDMAN  III  database.  Although  allowances 
could  be  made  to  approximate  this  new  task,  wide  variations  in  output  could  occur. 

3.6  ASSESSMENT  OF  HARDMAN  111  SOFTWARE 

3.6.1  Distribution  of  HARDMAN  Til  Software.  The  HARDMAN  III  was  initially 
distributed  exclusively  through  several  ARI  field  offices  at  Forts  Rucker,  Bliss,  and 
Sill.  These  field  offices  have  applied  the  software  to  several  "notionar  Army 
materiel  systems  (e.g..  Patriot  Navigation  Emplacement  System  (PNAVES),  AFAS, 
and  Forward  Area  Refueling  Point  (FARP)).  Several  individuals  involved  in  these 
tests  were  surveyed  as  SMEs  (see  Appendix  E). 

3.6.2  Lack  of  Technical  Specifications.  There  were  three  different  contractors  - 
Dynamics  Reseach  Corporation  (DRC),  Micro  Analysis  &.  Design  (MAD)  and  Applied 
Science  Associates  (ASA)  who  were  involved  in  the  design  and  development  of  the 
HARDMAN  III  software  modules.  The  complexity  of  managing  three  different 
contractors  may  have  led  to  the  diverse  software  schemes  used  by  each  of  the 
contractors  such  as  use  of  the  same  key  strokes,  menus  and  procedures,  etc.  Detailed 
technical  specifications  could  have  demanded  standardized  features  like  the  family  of 
Macintosh  software  (e.g.,  commonalities  with  regard  to  menus,  screen/window  layout 
and  routine  functions). 

3.6.3  Utility  Assessment.  The  utility  assessment  was  accomplished  in  two  stages: 
module  familiarization  and  evaluation.  The  assessment  was  performed  on  a 
combination  of  HARDMAN  III  software  versions  1  and  2  using  various  releases  (see 
Table  3-1).  T-CON  provided  no  version  disclosure  screen  and  therefore  no  version 
information  was  available  on  this  module.  Module  familiarization  consisted  of 
learning  the  basics  of  HARDMAN  III  software  operations.  Installation  time  and 
tutorial  time,  as  listed  in  Table  4-1,  would  be  a  subset  of  module  familiarization, 
since  the  time  listed  to  install  the  software  was  contingent  on  the  machine  speed  and 
the  time  to  run  through  the  tutorial  was  contingent  upon  the  amount  of  documentation 
provided.  Additional  time  was  required  to  gain  an  understanding  of  installation  and 
tutorial  procedures  (not  broken  out  in  the  chart  but  discussed  in  more  detail  in 
subsequent  paragraphs).  Installation  procedures  involved  reviewing  the  documentation 
for  installation  steps  and  following  screen  directions.  Tutorial  steps  were  followed 
with  information  provided  by  the  documentation  augmented  by  AGS  COE  A  data.  An 
additional  aspect  of  module  familiarization  was  the  time  needed  to  perform  a 
HARDMAN  III  analysis  utilizing  AGS  data.  Evaluation  times  are  not  given  since  they 
have  no  impact  on  workload  for  users  in  the  field  (who  will  be  evaluating  materiel 
systems  and  not  software  systems). 
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Table  3-1 


HARDMAN  III  SOFTWARE  VERSIONS 


MODULE 


MANSEVAL: 


M-CON: 


P-CON: 


SPARC: 


FILE  TYPE  VERSION 

RELEASE 

DATE 

Program  Version 

2.6 

7/02/92 

Library  Database 

2.3 

3/26/92 

Work  Database 

2.2 

8/26/91 

Library  Database 

1.8 

3/26/92 

Work  Database 

1.5 

8/14/91 

Help  Database 

2.1 

8/08/91 

M-CON 

1.1 

2/24/92 

System  Working  Files 

1.0 

8/26/91 

M-CON  Library 

1.13 

2/24/92 

Flow  Model  Library 

1.03 

2/24/92 

P-CON 

1.10 

2/24/92 

System  Working  Files 

1.00 

8/26/91 

P-CON  Library 

1.13 

2/24/92 

Flow  Model  Library 

1.03 

2/24/92 

SPARC  Program  Version 

2.2 

3/20/92 

SPARC  PRIWRK  Database  Version 

2.1 

8/08/91 

SPARC  PRILIB  Database  Version 

2.3 

4/15/92 

SPARC  PRIHLP  Database  Version 

2.1 

8/08/91 

PERSEVAL 

1.04 

2/20/92 

System  Working  Files 

1.00 

8/26/91 

PERSEVAL  Library 

1.1 

12/18/91 

3.6.4 


3.6.4. 1  Versions.  The  HARDMAN  III  versions  that  we  evaluated  were  obtained  from 
ARL  during  the  month  of  July  1992.  The  contractor  provided  a  total  of  forty-six  (46) 
3  1/2"  micro  floppy  disks  (at  no  cost  to  the  government)  to  ARL,  who  performed  the 
diskcopy.  After  receiving  the  disks,  they  were  loaded  on  an  IBM  compatible  with  an 
80386  microprocessor  and  an  80  megabyte  (MB)  hard  disk  drive.  Total  bytes  installed 
were  45,087,026  or  45  MB,  of  which  25,568,391  bytes  were  R:Base  files  (56.71%  of 
the  total)  (see  Table  3-2).  The  remaining  19,518,635  bytes  were  executable  files  and 
other  file  types  that  were  not  identified. 

3. 6. 4. 2  Workload.  A  total  of  75  minutes  (machine  time)  was  required  to  install  the 
software.  An  additional  15  minutes  was  required  per  module  to  read  the 
documentation  and  initiate  the  installation  process.  The  total  installation  time 
amounted  to  three  hours.  There  were  two  distinct  installation  packages:  one  package 
shared  by  ASA  and  MAD  for  T-CON,  SPARC  and  MAN-SEVAL,  and  one  provided 
by  DRC  for  M-CON,  P-CON  and  PER-SEVAL  installation.  Since  the  installation 
packages  developed  for  MAD  and  ASA  products  were  alike  in  form  and  function,  no 
distinction  will  be  made  between  them.  The  MAD  and  ASA  installation  screens  were 
graphical  in  nature  and  very  user  friendly.  There  was  little  narrative  documentation 
provided  and  installation  steps  were  provided  on  the  screen.  Prompts  furnished 
guidance  for  the  sequence  of  disk  insertion.  Anyone  with  minimal  MS-DOS 
knowledge  could  perform  this  installation,  e.g.,  an  administrative  technician,  a  junior 
staff  analyst,  etc.  The  installer  program  offers  ample  opportunity  to  escape  out  if  the 
user  makes  a  mistake  and  needs  to  cancel  a  command.  This  option  is  very  useful 
during  the  initial  installation.  The  DRC  installation  screen  was  not  as  graphical  as 
the  ones  provided  by  MAD/ASA  and  lacked  the  user  friendliness  normally  associated 
with  a  graphical  display.  DRC  provided  less  narrative  documentation  compared  to 
MAD/ASA.  The  user  must  be  very  careful  operating  the  DRC  installation  screens 
since  the  disk  prompts  are  slightly  confusing  and  must  be  very  careful  and  follow  the 
directory  prompt  when  it  asks  for  a  particular  disk  or  to  execute  a  particular 
command.  Since  the  installer  does  not  provide  the  user  with  an  escape  option  during 
a  function,  the  user  must  re-initiate  the  installation  process  if  a  significant  mistake 
occurs.  The  installer  of  DRC  programs  must  possess  a  working  knowledge  of  MS- 
DOS  and  make  error  free  procedural  steps  to  ensure  a  successful  installation.  DRC 
does  provide  a  de-installation  package,  in  addition  to  the  installation  package,  to 
correctly  back-up  the  module.  This  is  important  since  some  HARDMAN  III  files 
cannot  be  backed-up  onto  a  single  disk.  The  flow  model  contained  in  M-CON  and  P- 
CON  (over  3.5  MB)  provides  an  example.  When  there  are  breaks  in  large  files,  the 
software  must  be  told  where  the  breaks  are,  or  the  file  is  not  readable.  The  de¬ 
installer  conducts  this  record  keeping  function.  By  making  use  of  this  back-up 
package,  the  need  for  another  application  to  execute  this  function  is  eliminated. 
MAD/ASA  does  not  list  the  availability  of  this  function  in  its  reference  documentation. 


TABLE  3-2 
HARDMAN  III 

DATABASE  REQUIREMENTS 
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3. 6. 4. 3  Special  Problems  with  T-CON.  The  installer  modified  the  system 
configuration  file— with  unknown  effects  to  the  operating  system.  The  installer 
changed  the  CONFIG.SYS  files  line  to  equal  twenty  (apparently  T-CON  will  not  run 
reliably  with  fewer  than  twenty  files  specified  in  the  CONFIG.SYS  file);  also,  the 
installer  changed  the  AUTOEXEC.BAT  file  to  open  with  the  T-CON  directory  (not 
necessary  since  a  simple  C:CD\TCON  will  open  the  T-CON  directory).  The 
application  requires  very  close  to  640  K  available  random  access  memory  (RAM)  in 
order  to  operate,  which  is  difficult  to  obtain  on  some  machines  given  the  amount  of 
total  RAM,  the  size  and  location  of  MS-DOS,  and  the  existence  of  other  memory- 
resident  programs  (e.g.,  Microsoft  Windows  or  a  local/wide  area  network).  Most 
programs  are  designed  to  run  at  512  K  RAM,  with  the  remaining  space  allocated  for 
MS-DOS.  By  requiring  over  600  K  RAM,  a  hardware  problem  is  created  for  many 
users  since  there  may  be  insufficient  space  for  memory-resident  programs  such  as  a 
mouse  or  antiviral  software. 

3.6.5  Tutorials.  Each  contractor  provided  a  different  tutorial  with  a  different 
format.  The  amount  of  reference  documentation  varied,  as  well  as  the  training  style 
for  each  product.  The  tutorials  ranged  from  practically  non-existent  (ASA)  to  highly 
graphical  (DRC);  reference  documentation  ranged  from  very  little  (DRC)  to  excessive 
narrative  (MAD). 

3.6.S.1  Each  module  utilized  a  distinctive  presentation  style,  reflecting  the  different 
contractors  involved,  especially  with  respect  to  graphical  display,  format,  and 
presentation  of  instruction.  The  training  tutorial  provided  by  MAD  had  a  complex 
structure  that  would  be  difficult  for  the  average  analyst  to  quickly  grasp.  This 
presentation  style  was  characterized  by  mixing  reference  documentation  with  a  tutorial 
and  not  presenting  the  information  in  a  simple  and  straightforward  manner.  Type-in 
values  were  only  partially  provided  often  forcing  the  user  to  furnish  them  and 
increasing  the  required  execution  time.  The  MAD  product  tutorial  required  two  hours 
for  SPARC  and  four  hours  for  MAN-SEVAL,  respectively.  Do  to  the  complex 
structure  of  the  tutorial,  the  typical  user  would  find  it  difficult  to  form  a  gestalt  or 
overall  understanding.  Conversely,  DRC  provided  a  tutorial  using  a  very  graphical 
presentation  style,  but  lacked  a  good  written  description  of  the  procedures  contained 
in  the  module;  it  was  mostly  a  "structured  walk-through.”  The  tutorial  consisted  of 
a  screen  by  screen  presentation  of  the  module’s  capabilities.  "Type-in"  values  were 
provided  to  simplify  the  process,  eliminating  the  need  for  the  user  to  provide  figures, 
and  reducing  the  time  requirement  to  perform  the  tutorial  to  about  an  hour  per 
module.  The  lack  of  documentation  places  a  burden  on  the  user  to  learn  many  of  the 
higher  functions  without  the  benefit  of  a  written  description.  This  approach, 
sometimes  referred  to  as  the  "Video  Game"  approach  (discovering  higher  functions  by 
trial  and  error),  is  labor  intensive  and  requires  higher  analytical  skills,  knowledge, 
and  abilities.  There  is  the  added  problem  of  retaining  institutional  knowledge  with 
the  video  game  approach.  In  the  event  the  MPT  analyst  does  not  keep  adequate 


historical  records,  the  corporate  knowledge  may  be  lost  when  that  individual  rotates 
to  another  position.  Regardless  of  the  method  used  by  the  analyst  to  learn  the 
software,  good  reference  documentation  should  always  be  available. 

3. 6. 5. 2  ASA  did  not  provide  a  true  tutorial  for  T-CON.  All  that  was  included  in  the 
documentation  was  a  section  that  described  the  steps  in  T-CON.  No  "type-in"  values 
were  provided,  merely  a  discussion  of  the  process. 

3.7  HARDWARE  AND  SOFTWARE  REQUIREMENTS 

3.7.1  Hardware.  The  HARDMAN  III  modules  were  loaded  on  two  IBM  compatible 
PCs.  The  first  PC  used  an  80286  processor  with  640  K  bytes  RAM  expanded  to  1 
MB,  and  incorporated  an  Enhanced  Graphics  Adapter  (EGA).  MS-DOS  version  5.0 
was  loaded  into  high  memory  (640K  to  1  MB).  A  total  of  80  MBs  was  available  on 
the  hard  disk.  No  math  co-processor  was  present.  The  unit  had  two  high  density 
drives:  one  3-1/2"  and  one  5-1/4"  floppy  disk  drive.  The  second  PC  had  a  80386 
processor  with  4  MB  RAM,  and  operated  at  33  mhz.  The  operating  system  was  the 
MS-DOS  Version  5.0  and  used  the  Quarterdeck  Extended  Memory  Manager  (QEMM). 
It  had  an  80  MB  hard  disk  drive  with  two  high  density  floppy  drives:  one  3-1/2"  and 
one  5-1/4."  This  PC  utilized  a  Video  Graphics  Adapter  (VGA)  (see  Table  3-2  for 
database  requirements).  No  math  co-processor  was  present. 

3.7.2  Inconsistencies.  The  HARDMAN  III  modules  were  designed  to  be  consistent 
in  architecture  but  many  variations  exist  among  the  contractors’  modules.  The 
similarities  include  the  universal  use  of  the  FI  key  to  access  help  files;  each  product 
screen  provides  the  current  path  name  for  the  file  the  user  is  accessing  in  order  to 
establish  an  audit  trail;  also,  all  database  files  were  written  in  R:Base.  Other 
similarities  are  present,  but  the  following  include  some  of  the  dissimilarities.  One 
of  the  inconsistencies  involves  print  features.  DRC  programs  included:  a  function 
command  to  print  the  current  screen  (as  opposed  to  the  print  screen  command  which 
appears  on  the  keyboard,  limiting  printing  to  what  could  be  seen  on  the  screen). 
Although  this  is  a  very  useful  feature,  ASA  and  MAD  did  not  include  it  in  their 
programs,  allowing  only  report  and  graphical  printing.  DRC  provided  a  very  simple 
screen  to  set  print  features  (e.g.,  type  of  printer,  printer  port  designation, 
graphic/text,  etc.).  Neither  ASA  nor  MAD  had  this  feature.  DRC  provided  a  de- 
installation  application  with  their  modules  enhancing  back-up  procedures  whereby 
large  files  can  be  divided  among  several  disks.  ASA  and  MAD  provided  a  very 
graphical  installation  package  that  is  user  friendly,  while  DRC’s  displays  are  not  as 
user  friendly.  Some  inconsistencies  exist  among  replacement  systems  contained  in  the 
database  for  the  different  contractors.  For  example,  DRC  lists  the  system  name  for 
the  close  combat  light  rifle  as  the  M16AI;  MAD  lists  the  same  system  as  the  M16; 
ASA  lists  the  M16A2.  Also,  DRC  listed  the  M102  105  mm  with  the  fire  support 
towed  howitzer  system  group.  MAD  did  not  include  the  M102.  Therefore,  if  a  user 


needs  the  M102  for  a  replacement  system,  the  analyst  can  only  use  it  with  DRC 
products.  ASA’S  product  T-CON  is  very  different  than  the  other  modules.  It  contains 
34  mission  areas  compared  to  the  21  and  23  Battle  Functional  Mission  Areas  for  DRC 
and  MAD,  respectively.  T-CON  requires  a  great  deal  more  RAM  to  operate  than  the 
other  modules  and  will  not  run  reliably  with  less  than  640  K  bytes,  creating  an 
additional  hardware  problem  since  many  PCs  only  have  640  K  bytes  of  total  RAM. 
Out  of  this  640  K,  MS-DOS  must  be  used  as  a  memory-resident  program  in  order  for 
the  machine  to  operate.  Therefore,  it  is  difficult  for  T-CON  to  run  on  a  PC  with  less 
than  1  MB  random  access  memory.  Additionally,  it  takes  an  enormous  amount  of  time 
to  run  a  T-CON  evaluation  (e.g,  on  a  PC  with  a  80386  processor  it  took  1.5  hours  to 
run  an  analysis  and  over  seven  hours  for  a  80286  processor  equipped  PC).  Table  3-3 
recaps  the  hardware  inconsistencies  among  the  six  modules  as  a  result  of  having  three 
different  contractors. 

3.7.3  Linkages.  Linkages  are  inconsistent  prohibiting  free  transferability  of  work 
file  data  among  modules.  For  example,  SPARC  is  the  first  HARDMAN  III  product, 
but  output  from  SPARC  is  not  linkable  with  M-CON  -  the  second  product.  P-CON 
(a  DRC  product)  output  is  not  linkable  with  PER-SEVAL  (another  DRC  product), 
although  both  modules  relate  to  personnel  issues.  Since  many  modules  are  not 
linkable,  the  user  is  forced  to  populate  the  workfiles  of  various  modules  with  similar 
data,  creating  additional  workload  (see  Figure  3-1  for  depiction  of  linkages,  covering 
both  import/export  and  backup).  The  possibility  exists  that  an  expert  could  manually 
change  the  header  record  for  the  import  files  and/or  make  other  changes  as  necessary 
so  that  a  module  not  previously  configured  to  import/export  that  data  file  type  could 
read  it.  Regardless,  this  topic  is  not  covered  in  the  reference  documentation  and  the 
average  user  would  lack  the  knowledge  base  to  do  that  type  of  translation  unaided. 

3.7.4  Non-Standard  Screen  Features.  The  use  of  the  escape  key  differs  from 
module  to  module.  In  MAD  products,  it  is  possible  to  strike  the  escape  key  until  the 
user  has  exited  the  module  entirely.  DRC  products  allow  the  user  to  escape  back 
until  reaching  the  main  menu,  and  then  no  further.  Item  selection  procedures  differ 
from  contractor  to  contractor.  In  SPARC,  the  user  must  employ  a  combination  of 
arrow  keys  and  the  space  bar.  DRC  products  use  only  the  space  bar  (since  striking 
the  space  bar  highlights  a  selection  and  drops  to  the  next  line  down).  This  minor 
issue  points  out  the  larger  issue  of  inconsistency  and  how  it  impacts  workload.  The 
user  must  remember  all  the  small  differences  between  the  modules  each  time  a 
particular  module  is  accessed  involving  reviewing  notes,  documentation,  etc.  This 
increases  the  learning  curve  and  causes  additional  physical  and  cognitive  workload. 

3.7.5  Disk  Back-Ups  Differ  Among  Contractors.  When  the  user  backs  up  data 
onto  a  disk  using  DRC  generated  software,  the  application  automatically  erases  the 
destination  disk.  This  precludes  the  user  from  placing  multiple  back-ups  on  the  same 
disk.  MAD’s  back-up  utility,  on  the  other  hand,  does  not  erase  the  disk,  allowing  for 
multiple  back-ups  on  the  same  destination  disk,  which  reduces  overall  cost.  It 


Table  3  -  3 


Hardware  Configuration  Inconsistencies 


Processor 

80286 

Random  Access  Memory 

640K  Bytes 

Monitor 

ASA:  Monochrome  or  Color  Display 

DRC:  Not  Specific 

MAD:  Enhanced  Graphics  Display 

Disk  Drives 

Floppy  Diskette  or  Bernoulli  Cartridge 

Printer 

ASA:  Text  Printer 

DRC:  IBM  Compatible  Printer 

MAD:  Dot  Matrix,  IBM  graphics,  80  Characters 
per  inch 

DOS  Requirements 

ASA:  MS  DOS  Version  3.2  or  higher 
DRC:  MS  DOS  Version  3.0  or  higher 
MAD:  MS  DOS  Version  3.2  or  higher 


MAD 
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HARDMAN  III 
MODULE  LINKAGES 


Figure  3-1 


becomes  necessary  to  re-read  documentation  each  time  the  user  accesses  a  particular 
module  in  order  to  become  reacquainted  with  that  particular  module’s  functions  and 
input  procedures.  The  major  similarity,  among  the  modules,  is  that  they  are  all  menu 
driven  and  utilize  a  step-order  approach  to  organize  input.  However,  input  patterns 
differ  from  contractor  to  contractor.  This  contributes  to  the  major  problem  of  the 
modules  not  being  consistent  with  each  other.  This  non-standardization  requires 
additional  time  for  the  analyst  to  perform  the  steps  of  each  module  since  the  user 
must  remember  different  application  styles.  It  would  have  been  much  simpler  if  the 
modules  were  set  up  with  more  similarity  thus  improving  user  friendliness.  At  a 
minimum,  even  if  the  input  pattern  was  not  the  best,  the  user  would  only  have  to 
learn  it  once,  simplifying  input  operations. 

3.8  DOCUMENTATION 

All  HARDMAN  III  modules  came  with  documentation  (all  version  1.0),  but  the 
user  manuals  that  were  included  often  lacked  utility  or  provided  limited  utility.  This 
reduced  the  overall  usefulness  of  the  manual  as  a  complete  reference  tool. 

3.8.1  Incomplete  Manuals.  For  example,  only  MAD’s  user  manuals  contained  an 
index  to  facilitate  reference  searches.  ASA  included  a  list  of  acronyms  and 
abbreviations  but  did  not  include  a  glossary  of  term  definitions.  DRC  and  MAD,  on 
the  other  hand,  did  include  a  glossary  of  term  definitions,  but  did  not  include  a  list 
of  acronyms  and  abbreviations.  DRC  lists  twelve  mission  areas  in  the  software  and 
in  the  documentation,  although  only  six  can  be  accessed  in  the  module  (which  is  not 
mentioned).  Also,  the  system  types  that  make  up  these  mission  areas  are  not  listed 
at  all.  MAD  does  not  list  the  mission  areas  or  the  system  types  in  the  documentation. 
ASA,  on  the  other  hand,  lists  all  system  types  by  mission  area  in  the  user  manual. 
The  user  manual  version  is  well  behind  the  current  software  versions.  MAN-SEVAL 
and  SPARC  are  well  into  their  second  versions;  M-CON,  P-CON  and  PER-SEVAL 
have  had  many  releases,  which  may  include  changes  that  are  not  reflected  in  the 
reference  documentation.  The  reference  manual  documentation  is  laid  out  in  a  step 
approach,  exactly  as  it  is  in  the  software.  This  is  useful  in  order  to  illustrate  the 
process  involved  in  performing  HARDMAN  III  analysis,  but  it  is  very  limiting  as 
well.  Once  an  analyst  has  gone  through  the  steps  several  times,  he  or  she  will 
become  accustomed  to  the  overall  approach  and  will  not  need  to  repeat  the  tutorial  in 
all  future  evaluations.  This  would  imply  that  a  more  topical  approach  to  reference 
source  arrangement  would  be  in  order  since  the  analyst  would  need  to  become 
reacquainted  with  specific  functions  and  not  the  flow  process  in  its  entirety. 

3.9  HELP  SCREENS 

Much  of  the  documentation  available  to  assist  the  user  is  contained  in  an 
assortment  of  help  menus.  These  help  menus  are  easily  accessible,  but  often  are 
limited  to  basic  guidance  (e.g.,  defining  simple  function  commands).  Additionally, 


the  software  does  not  document  algorithms  well,  by  way  of  screen  alerts  and 
messages.  Often,  the  user  does  not  know  what  goes  on  inside  the  computer  when  the 
screen  says  "wait".  This  can  be  troublesome  when  lengthy  computations  are  being 
conducted. 

3.9.1  Incomplete  Information  About  Critical  Items.  For  example,  in  the  P-CON 
screen,  "Assess  MOS  Availability  and  Performance,"  under  edit  task,  the  help  screen 
only  defines  three  taxons,  although  there  are  nine  in  total.  Note:  a  taxon  is  a  task 
classification  category.  For  example,  a  visual  task  requires  using  the  eyes  to  identify 
targets,  a  numerical  task  requires  performing  mathematical  calculations,  and  a 
cognitive  task  requires  processing  information  mentally  and  reaching  a  conclusion. 
It  would  be  very  helpful  to  the  user  to  have  all  taxons  defined  since  they  must  be 
independently  attributed  to  specific  tasks.  Given  the  fact  that  the  user  must  make  a 
fairly  subjective  determination  of  taxon  application,  good  definitions  are  crucial. 
Although  help  screens  provide  some  reference  utility,  their  use  does  not  replace  good 
user  guide  documentation. 

3.10  DATABASE  CONFIGURATION 

Configuration  control  of  various  versions  of  HARDMAN  III  could  present  a 
problem  given  the  number  of  separate  modules  with  their  different  library  databases, 
each  containing  varying  degrees  of  conformance.  This  would  constitute  an  additional 
workload  for  TRADOC  Headquarters  to  maintain  control  of  the  various  versions  of 
the  HARDMAN  III  data  libraries. 

3.10.1  Differing  Versions.  There  are  many  different  versions  of  the  library 
database  (see  Table  3-1).  The  effective  dates  of  the  libraries  range  from  PER- 
SEVAL  (December  1991)  to  SPARC  (April  1992).  These  databases  contain  different 
versions  of  weapon  systems  (as  depicted  earlier  in  reference  to  the  M16  rifle)  and  not 
all  modules  have  the  same  replacement  system  types  (MAD  does  not  include  the 
M102,  but  DRC  does).  The  output  HARDMAN  III  produces  is  a  result  of  information 
contained  in  the  library  database.  As  the  user  proceeds  from  one  module  to  the  next, 
database  inconsistencies  could  distort  overall  output.  This  lack  of  uniformity  is  due 
to  each  module  using  the  same  replacement  system  types  in  different  configurat^'ons. 

3.10.2  Redundancies  in  the  Database.  The  replacement  system  types  are  included 
(in  one  form  or  another)  in  all  the  modules,  requiring  a  great  deal  of  disk  space  to 
accommodate,  and  are  listed  in  Table  3-4.  Possible  solutions  to  the  problems  of 
uniformity  and  redundancy  are  covered  in  subsequent  sections.  Also,  a  lot  of  data  the 
user  inputs  into  one  module  is  similar  to  data  that  has  been  input  into  other  modules, 
requiring  additional  workload. 

3.10.3  Limitations  in  the  Library.  There  are  some  limitations  in  the  library 
database,  since  only  six  mission  areas  are  included.  DRC  lists  an  additional  six 
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Table  3-4 


Hardman  in  Mission  Areas 


^  MISSION  AREA 

SYSTEM  TYPE 

SYSTEM  name'* 

Air  Defense 

HIMAD 

Patriot 

Man-Portable  Systems 

Stinger 

Mobile  Gun  Systems 

Vulcan 

Aviation 

Attack  Helicopter 

AH-64 

Cargo  Helicopter 

CD-47D 

Scout  Helicopter 

OH-58D 

Utility  Helicopter 

UH-60 

Close  Combat-Heavy 

Cavalry  Fighting  Vehicle 

M3 

Tanks 

Ml 

Close  Combat-Light 

Anti-Tank  Vehicle 

ITV 

Anti-Tank  Weapon 

Dragon 

Grenade  Launcher 

M-203 

Infantry  Fighting  Vehicle 

M2 

Machine  Guns/Automatic 

SAW 

Man-Portable  Indirect 

81mm  Mortar 

Rifles 

M-16 

Combat  Service  Support 

Heavy  Cargo  Trucks 

HEMTT 

Light  Cargo  Trucks 

HMMWV 

Command  and  Control 

No  system  type  data 

Communications 

No  system  type  data 

Engineering  and  Mine  Warfare 

No  system  type  data 

Fire  Support 

Medium  range  Missile 

LANCE 

Rocket  Systems 

MLRS 

Self-Propelled  Howitzer 

M-109 

Towed  Howitzer 

M-198 

Intelligence  and  Electronic  Warfare  No  system  type  data 
Nuclear,  Biological  and  Chemical  No  system  type  data 
Special  Operations  No  system  type  data 


mission  areas  that  included  no  system  type  data  (as  listed  in  Table  3-4).  These  are 
Command  and  Control;  Communications;  Engineering  and  Mine  Warfare;  Intelligence 
and  Electronic  Warfare;  Nuclear,  Biological  and  Chemical;  and  Special  Operations. 
If  these  mission  areas  and  related  system  types  were  included  in  each  library  database, 
the  total  disk  requirement  would  be  substantially  larger.  An  additional  cost  would  be 
incurred  by  TRADOC  to  populate  each  database  with  this  and  other  new  data. 

3.10.4  Configuration  Control.  Due  to  the  lack  of  configuration/version  control, 
there  could  be  wide  variations  in  the  reliability  of  the  individual  HARDMAN  III 
libraries,  and  it  would  require  additional  TRADOC  manpower  to  manage  them,  in 
terms  of  verification  and  validation,  since  each  library  for  each  module  would  have 
to  be  independently  reviewed. 

3.10.5  Use  of  RrBase  Software.  HARDMAN  III  modules  include  R;Base  data  files 
comprising  the  Mission  Area  data  library  (see  Table  3-2  listing  R:Base  file  types  by 
functional  category).  HARDMAN  III  was  designed  so  that  users  would  not  need  the 
R;Base  application  to  access  data  libraries.  Although  a  licensed  copy  of  the 
application  is  not  distributed  with  the  modules,  copyright  obligations  still  must  be 
satisfied  with  respect  to  the  R:Base  generated  data  files.  This  is  achieved  by 
purchasing  the  application  Run  Time  during  product  design,  which  acts  as  a  licensing 
vehicle  that  can  be  used  without  distribution  restrictions.  Run  Time  is  essentially  an 
R:Base  application  that  lacks  the  R:Base  engine,  and  so  data  element  editing  functions 
are  not  possible.  It  consists  of  a  command  file  and  three  (3)  RBF  files.  But,  it  is 
necessary  to  have  the  full  blown  application  present  if  the  user  or  configuration 
manager  needs  to  manipulate  library  data  elements.  For  example,  the  application  must 
be  present  to  update  existing  data  files  or  to  create  a  new  system  database.  Since 
R.'Base  is  not  in  the  public  domain,  an  additional  cost  could  be  incurred  for  the 
purchase  of  licensed  copies  and  to  upgrade  to  higher  versions.  This  presents  a 
potential  problem  for  the  government  in  the  event  TRADOC  distributed  unlicensed 
copies  of  the  copyrighted  software  such  as  R:Base  or  MicroSaint  (see  below). 

3.10.6  Use  of  MicroSaint  Software.  MicroSaint  is  a  PC-based  discrete  event 
simulation  modeling  tool  used  in  several  of  the  HARDMAN  III  modules  (SPARC, 
MAN-SEVAL  and  PER-SEVAL)  for  replicating  maintenance  scenarios  and  predicting 
RAM  results.  The  developers  of  microSaint  (MAD)  have  given  the  government 
unlimited  access  to  the  use  of  the  codes  including  approval  to  duplicate  copies. 
Therefore,  there  is  no  problem  of  committing  the  government  to  a  sole  source  use  of 
this  software.  However,  these  privileges  have  not  been  extended  to  commercial  firms. 


3.10.7  Availability  of  Software  From  the  Defense  Technical  Information  Center 
fPTlC).  There  have  been  several  advertisements  in  the  MANPRINT  bulletin  stating 
that  the  HARDMAN  III  software  was  available  from  DTIC.  An  independent  request 
for  the  modules  was  made  which  prompted  a  number  of  questions  back  from  DTIC. 
The  end  result  is  that,  as  of  the  publishing  of  this  report,  it  will  take  several  more 


months  to  process  and  catalog  the  HARDMAN  III  software  program  for  eventual 
distribution  to  DoD  users. 


3.11  AUDIT  TRAIL 

The  author  tended  to  become  disoriented  within  a  given  HARDMAN  III  module. 
Although  a  pathname  is  provided  at  the  top  of  each  menu,  it  occasionally  was  unclear 
how  the  pathname  related  to  the  main  menu. 

3.11.1  Screens.  During  some  steps  in  HARDMAN  III  modules,  input  screens  are 
layered  in  order  for  the  user  to  graphically  determine  where  they  are.  This  occurs 
frequently  in  the  "Identify  Systems  to  be  Replaced"  step.  The  application  (DRC  in 
particular)  layers  screens  in  a  diagonal  pattern  that  is  easy  for  the  user  to  follow. 
But,  this  does  not  occur  in  all  cases.  Sometimes  in  more  detailed  steps  that  require 
numerous  sub-steps,  it  is  easy  for  the  user  to  become  disoriented,  in  light  of  the 
assortment  of  escape  commands  and  menu  commands  that  are  necessary  to  enter  data 
or  return  to  the  main  menu. 

3.12  NEW  MOSs 

It  is  often  necessary  to  create  a  new  MOS  for  a  new  materiel  system. 
HARDMAN  III  does  not  allow  the  MPT  analyst  to  create  new  MOSs  in  the  event  that 
new  and  revolutionary  technology  requires  increased  personnel  attributes  or  skill 
levels.  This  situation  might  be  encountered  when  manual  tasks  are  replaced  by 
automated  tasks.  Because  HARDMAN  III  utilizes  MOSs  that  are  included  in  a  static 
database,  it  is  difficult  for  the  analyst  to  evaluate  new  materiel  systems  that  might 
call  for  new  MOSs,  unless  the  analyst  had  access  to  the  data  libraries  through  R:Base. 
With  the  presence  of  the  application  R:Base,  the  analyst  should  have  access  and  be 
able  to  make  modifications  to  the  library  data  files.  This  would  require  additional 
knowledge  and  reference  documentation  describing  how  this  procedure  is 
accomplished. 

3.13  HARDMAN  III  Software  Enhancements 

Two  of  the  contractors  (DRC  and  MAD)  that  designed  and  developed  five  of 
the  six  HARDMAN  III  software  modules  (T-CON  being  the  exception)  have  previously 
identified  enhancements  to  improve  HARDMAN  III.  These  enhancements  are 
contained  in  section  5  of  the  "Final  Report  for  Concepts  on  MPT  Estimation",  dated 
December  1990  (see  Appendix  B). 

3.14  COST  AND  MANPOWER  REQUIREMENTS 

The  costs  for  operating  and  maintaining  the  HARDMAN  III  software  for 
TRADOC  and  its  schools,  centers  and  field  operating  agencies  are  broken  down  as 
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follows: 


(a)  Initial  and  recurring  workload  costs  -  estimated  at  approximately  at  least 
four  weeks  to  load  the  initial  software  module  and  become  proficient  in  its  use  (see 
Table  3-5). 

(b)  It  appears  that  the  computer  hardware  and  software  requirements  needed 
to  run  HARDMAN  III  may  have  been  underestimated  by  the  developing  contractors. 
Table  3-6  lists  the  initial  hardware  requirements  to  properly  operate  the  HARDMAN 
III  modules. 

(c)  Initial  and  recurring  training  costs  -  approximately  two  weeks  annually  of 
training  time  is  needed  to  attend  the  initial  course  and  to  undergo  periodic  refresher 
courses  to  overcome  the  high  decay  rates  inherent  in  the  use  of  the  HARDMAN  III 
software. 

(d)  Database  operation  and  support  (O&S)  costs.  HARDMAN  III  software 
configuration  control  would  require  one  full  time  position  and  approximately  $12,000 
per  quarter  to  maintain  a  centralized  database  (high  fixed  -  low  maintenance  costs)  if 
the  HARDMAN  III  configuration  were  to  be  controlled  and  maintained  by 
Headquarters  TRADOC.  The  workload  associated  with  the  configuration  control  and 
standardization  requirements  as  new  versions  are  released  appears  to  be  labor 
intensive  (see  Figure  3-2  and  Table  3-5). Note:  the  projected  HARDMAN  O&S  costs 
were  derived  from  the  latest  FOOTPRINT  relational  database  costs.  The  configuration 
management  costs  of  maintaining  the  HARDMAN  III  modules  and  databases  could  be 
expensive.  Passing  information  via  modem  from  a  centralized  HQ  TRADOC  location 
to  individual  schools  could  be  time  consuming  (PC  downtime  during  transmissions) 
and  costly  (purchase  of  modems  and  peripheral  software  and  materials).  Conversely, 
if  each  TRADOC  school  was  assigned  responsibility  for  its  own  PC  database 
configuration  control  (low  fixed  -  high  maintenance  costs)  the  costs  would  be 
significantly  higher  since  each  school  would  require  one  full  time  HARDMAN  III 
analyst  performing  configuration  control  as  an  additional  duty.  In  addition,  each 
TRADOC  MPT  analyst  will  require  additional  1  megabyte  of  disk  storage  capability 
to  accommodate  the  HARDMAN  III  database  requirements.  This  additional  disk 
storage  space  requirement  can  be  satisfied  by  acquisition  of  removable  hard  disk 
drives  (e.g.,  Bernoulli  box  or  Quantum  Passport  XL  removable  disk  drive).  Cost  is 
$500.00  for  a  Passport  XL  installation  kit  (one  per  PC)  and  $800.00  per  disk  drive. 

(e)  An  additional  intangible  requirement  is  the  need  to  have  TRADOC  MPT 
analysts  possess  a  high  degree  of  prerequisite  analytical  skills,  knowledge  and  abilities 
(see  section  S)  to  become  qualified  in  the  HARDMAN  III  software. 

(f)  Classified  hardware  and  software  protection  costs  -  PCs  may  need  special 
protection  by  having  a  removable  hard  disk  drive  capability  and  classified  safe  in  the 
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event  it  is  necessary  to  use  and  analyze  classified  HARDMAN  III  data.  In  addition, 
a  relational  database  management  system  software  may  be  necessary  to  protect  data 
during  storage,  manipulation  and  retrieval  via  a  local  area  network. 

(g)  The  configuration  manager  should  be  very  careful  making  any  changes  to 
the  structure  of  the  data  files,  which  could  include  changing  data  length  or  data  type. 
A  change  made  in  one  data  file  could  affect  other  files,  including  executables,  since 
the  database  is  interrelated.  These  structure  changes  do  not  necessarily  have  to  be 
made  at  the  designer  site,  but  could  be  made  at  Headquarters,  TRADOC  if  the 
configuration  manager  has  all  the  codes  and  applicable  documentation  available. 
Making  changes  only  to  the  data  elements  should  not  be  a  problem,  but  care  must  be 
taken  if  any  other  files  might  be  affected.  It  is  also  important  that  the  configuration 
manager  be  a  highly  knowledgeable  systems  analyst  in  order  to  adequately  perform 
these  functions.  This  would  increase  the  total  cost  since  it  would  not  be  possible  to 
assign  these  configuration  management  functions  to  an  unqualified  employee. 

(h)  The  above  costs  could  be  exacerbated  by  the  increasingly  higher  annual 
turnover  and  attrition  in  "green  suit"  military  and  Department  of  the  Army  civilian 
MPT  analysts.  This  is  a  common  phenomenon  that  is  occurring  more  frequently  in 
the  TRADOC  Headquarters,  schools  and  agencies  as  the  U.S.  Army  downsizes  over 
the  next  5  years. 

The  HARDMAN  III  O&S  costs  for  Headquarters,  TRADOC  and  its  schools, 
centers  and  field  operating  agencies  appear  to  be  substantial  (see  Table  3-7).  These 
costs  assume  that  all  HARDMAN  III  work  will  be  performed  in-house  by  DA  analysts 
or  military  personnel.  The  configuration  management  assumes  that  all  the  codes  and 
applicable  documentation  are  available.  Making  changes  only  to  the  data  elements 
should  not  be  a  problem,  but  care  must  be  taken  if  any  other  files  might  be  affected. 
It  is  also  important  that  the  configuration  manager  be  a  highly  knowledgeable  systems 
analyst  in  order  to  adequately  perform  these  functions. 
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major  TRADCX!  schools  (estimated  10  sites). 


SECTION  4.0 

FINDINGS  AND  CONCLUSIONS 


4.0  FINDINGS  AND  CONCLUSIONS 

The  findings  and  conclusions  discussed  below  are  based  on  the  HARDMAN  III 
surveys,  discussions  with  MPT  SMEs  from  a  wide  section  of  the  Army,  attendance  at 
the  three-day  HARDMAN  III  training  seminar,  comparison  of  HARDMAN  III  with 
AGS  COEA  MPT  methodology,  and  evaluation  of  the  HARDMAN  III  software 
modules. 

4.1  FINDINGS 

4.1.1  HARDMAN  III  is  a  Misnomer.  Several  of  the  six  modules  do  not  enable 
an  MPT  analyst  to  calculate  a  materiel  system’s  manpower  and  training  resource 
requirements  data  that  previously  could  be  determined  using  the  HCM.  The  HCM 
allows  an  MPT  analyst  to  employ  several  methods  ranging  from  the  manual  "stubby 
pencil"  method  to  the  Digital  Equipment  Corporation  (DEC)/VAX  mainframe  based 
Man-Integrated  System  Technology  (MIST)  (HARDMAN  II)  software,  and  the  recent 
personal  computer  software  versions  (HARDMAN  II. 2  and  II. 3)  to  produce  a  myriad 
of  MPT  reports.  These  typical  HARDMAN  products  are  then  used  as  source 
documents  used  to  feed  and  support  key  milestone  decision  review  documents  such  as 
the  Manpower  Estimate  Report  (MER),  the  COEA,  the  System  MANPRINT 
Management  Plan  (SMMP),  the  Baseline  Cost  Estimate  (BCE),  and  Cost  Analysis 
Requirements  Document  (CARD).  Note:  Most  HARDMAN  III  surveyees  were  not 
aware  that  HARDMAN  III,  as  a  performance  measurement  tool,  is  considerably 
different  from  the  generic  HARDMAN  methodology.  Notwithstanding  the  apparent 
confusion  over  HARDMAN  III,  a  more  crucial  issue  is  whether  the  HARDMAN  III 
modules  can  be  used  to  produce  MPT  requirements  after  the  enhancements  are  made. 

4.1.2  MANPRINT  Concent  Not  Applied.  MANPRINT  considers  soldier 
performance  and  reliability  early  in  the  materiel  development  and  acquisition  process. 
MANPRINT  ensures  that  future  materiel  acquisitions  will  incorporate  design  features 
that  allow  a  greater  cross  section  of  our  projected  soldier  target  audience  force  to 
successfully  operate  and  maintain  the  proposed  equipment.  In  the  case  of  fielding  the 
HARDMAN  III  software,  the  MANPRINT  concept  should  have  been  employed  by  ARL 
by  adequately  researching  the  intended  target  audience  (e.g.,  TRADOC  analysts)  and 
performing  a  needs  analysis  of  what  the  ultimate  user  wanted  and  needed  to  better 
perform  a  MPT  analyses.  A  suggested  approach  would  have  been  to  have  a  series  of 
in-process  reviews  with  a  technical  advisory  group  consisting  of  a  cross  section  of 
MPT  analysts  to  get  early  SME  involvement.  This  approach  may  have  prevented  most 
of  the  user  friendliness  problems  inherent  in  the  HARDMAN  III  software.  The 


ultimate  question  for  the  HARDMAN  III  user  is  -  can  this  TRADOC  analyst,  with  this 
HARDMAN  III  training,  perform  the  desired  MPT  analytical  tasks,  to  the  TRADOC 
school  standards,  under  the  current  TRADOC  DCD  and  DOTD  school  conditions?  The 
answer  appears  to  be  negative  because  the  HARDMAN  III  software  did  not  undergo 
the  recommended  robust  MANPRINT  process  that  could  have  prevented  most  of  the 
problems  experienced  by  several  TRADOC  users  and  expressed  via  the  HARDMAN 
III  surveys.  The  HARDMAN  III  software  should  have  been  "MANPRINTED"  using 
the  appropriate  military  standards  and  Army  guides  to  ensure  that  prospective  Army 
analysts  could  perform  an  accurate  MPT  analysis  in  the  most  cost  effective  manner. 

4.1.3  Project  A  Database  mav  be  Outdated.  The  performance  data  extracted 
from  Project  A  forms  the  basis  for  the  HARDMAN  III  performance  data  since  it  was 
considered  the  best  available  data  at  the  time.  HARDMAN  III  uses  the  ASVAB  data 
to  predict  task  performance  of  the  available  soldier  population,  gauge  the  effects  of 
operating  and  maintaining  materiel  systems  varying  quality  of  soldiers,  and  soldier 
aptitude  requirements  for  existing,  developing,  and  non-development  item  (NDI) 
systems.  A  major  difficulty  with  using  the  Project  A  database  is  the  fact  that  recent 
accessions  of  higher  quality  soldiers  over  the  past  five  years  perhaps  have  rendered 
the  Project  A  database  out  of  date  and  could  be  misleading  since  the  original 
performance  data  represents  a  higher  percentage  of  non-high  school  graduates 
(Category  III  and  IV  individuals)  in  the  21  MOSs  tested  than  are  currently  being 
recruited.  This  skewed  data  might  be  acceptable,  but  current  validity  could  be  in 
question  since  it  would  be  difficult  to  remove  the  bias.  Also,  there  is  a  validity 
question  concerning  the  use  of  SQT  scores  as  an  indicator  of  performance  potential 
for  Project  A.  Project  A  may  also  be  erroneous  from  the  standpoint  that  personnel 
performance  characteristics  may  not  follow  linear  patterns,  especially  when  multiple 
stressors  are  encountered.  Finally,  there  is  the  questionable  validity  of  extrapolating 
performance  data  from  21  core  MOSs  to  more  than  2S0  additional  MOSs.  Use  of  such 
specious  data  to  predict  performance  capabilities  and  evaluate  stressors,  especially 
when  projecting  into  the  late  1990s  and  beyond,  could  provide  misleading  or 
erroneous  data  to  U.S.  Army  analysts. 


4.1.4  Evaluation  of  Trainin2.  Although  there  was  no  intent  to  design  and  develop 
the  HARDMAN  III  training  seminar  in  accordance  with  MIL-STD-1379D  (the  guiding 
directive  for  military  training  programs),  the  HARDMAN  training  appears  to  be 
inadequate  to  properly  train  and  qualify  a  typical  Army  MPT  analyst  as  a  HARDMAN 
III  analyst.  An  improved  tutorial  inherent  in  the  software  modules  would  be  a 
significant  improvement  and  should  eventually  replace  the  three-day  seminar.  The 
improved  HARDMAN  III  tutorial,  supplemented  by  a  demonstration  program  to  "walk 
the  analyst"  through  a  typical  HARDMAN  III  problem,  would  be  a  more  cost  effective 
way  of  training  new  HARDMAN  III  analysts  and  overcome  the  inadequacies  of  the 
training  seminar.  Also,  any  formal  training  course  should  only  be  developed  after 
the  target  audience  has  been  identified  and  defined. 
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4.1.5  Evaluation  of  HARDMAN  111  Software.  The  following  findings  were 
determined  as  part  of  the  HARDMAN  III  utility  Assessment. 

4.1.5.1  An  evaluation  of  user  interface  with  respect  to  the  module  reference 
documentation  and  application  screens  was  conducted.  During  the  evaluation,  many 
inconsistencies  were  uncovered  revealing  a  lack  of  compliance  with  DOD-STD-2167A, 
"Defense  System  Software  Development." 

(a)  There  are  non-standard  tutorial  documentation  among  modules.  The 
tutorial  documentation  for  each  module  used  different  styles,  reflecting  the  different 
contractors  involved,  especially  with  respect  to  the  graphical  displays,  format,  and 
presentation  of  instruction.  The  products  provided  by  MAD  have  a  complex  structure 
that  is  difficult  to  understand  for  the  average  analyst.  By  the  same  token,  DRC 
provided  a  very  graphical  tutorial  presentation  style,  but  lacked  a  good  written 
description  of  the  sequence  of  events.  Given  the  complexities  inherent  in  the  software 
modules,  some  type  of  training  should  be  included  with  each  module. 

(b)  There  is  limited  or  incomplete  HARDMAN  III  documentation.  For 
example,  the  "Structured  Walk-Through"  tutorial  provided  in  the  DRC  products  lacked 
sufficient  information  for  the  user  to  grasp  in  a  timely  fashion  and  apply  to  different 
situations.  This  contributes  to  a  lack  of  understanding  of  where  one  is  in  the  process 
since  there  is  no  path  or  logic  trail  to  follow. 

(c)  It  appears  that  linking  all  modules  was  a  fundamental  error.  An 
erroneous  assumption  was  made  that  all  users  would  be  using  all  modules.  This  is  not 
the  case,  as  borne  out  by  the  HARDMAN  III  surveys.  Most  responders  indicated  that 
one  or  possibly  two  modules  would  be  useful  to  their  MPT  analytical  needs.  Again 
a  thorough  needs  analysis  would  have  identified  the  target  audience  and  the  necessary 
analytical  tools  necessary  for  TRADOC  MPT  analysts  to  perform  their  analyses. 

(d)  There  was  an  absence  of  technical  specifications  from  the  government 
to  the  contractors  requiring  the  HARDMAN  III  modules  to  be  standardized.  These 
specifications  could  have  demanded  that  the  same  scheme  be  employed,  such  as  the 
same  key  strokes,  menus,  procedures,  etc.,  with  regard  to  menus,  screen/window 
layout,  and  routine  functions,  etc. 

4. 1.5.2  The  AGS  COEA  study  was  provided  by  the  government  as  a  reference  for 
the  analysis  using  HARDMAN  III.  The  AGS  information  was  provided  in  a  COEA 
format  that  makes  translation  into  the  HARDMAN  III  format  difficult.  The  main 
driver  behind  reducing  the  AGS  operator  crew  size  from  four  to  three  was  the  advent 
of  an  autoloader,  which  is  not  a  task  included  in  the  HARDMAN  III  database. 
Although  allowances  were  made  to  approximate  this  and  several  other  new  tasks,  wide 
variations  in  output  occurred  and  were  observed. 


4. 1.5.3  Configuration  control  inherent  in  managing  and  monitoring  various 
versions  of  HARDMAN  III  could  present  a  problem  given  the  number  of  separate 
modules  with  their  different  library  databases  each  containing  varying  degrees  of 
conformance.  This  would  constitute  an  additional  workload  for  Headquarters 
TRADOC  to  maintain  control  and  standardization  of  the  various  versions  of  the 
HARDMAN  software  modules  especially  as  additional  capabilities  like  Force  Analysis 
Aid  (FORCE),  Human  Operator  Simulator  (HOS),  and  the  Manpower  Capability  Aid 
(MANCAP)  modules  are  added  at  a  later  date. 

4.1.6  Positive  Aspects  of  HARDMAN  III.  There  is  some  utility  in  the  selected 
use  of  the  HARDMAN  III  modules  and  these  are  as  follows: 

4. 1.6.1  The  SPARC  database  provides  a  compendium  of  databases  for  21  mission 
areas  that  provides  Army  MPT  analysts  the  ability  to  research  a  database  from 
"scratch."  Updating  the  SPARC  database  with  the  latest  RAM  data  then  provides  the 
analyst  with  a  substantial  time  savings  in  constructing  the  first  generation  of  mission 
area  data. 

4. 1.6.2  The  MAN-SEVAL  and  PER-SEVAL  modules  are  excellent  tools  that  can 
be  used  to  evaluate  the  relative  merits  of  materiel  systems.  People  limiting  factors 
(e.g.,  taxons  such  as  motor  functions,  continuous  versus  discrete  tasks  and  climate 
stressors  (e.g.,  cold,  heat  and  noise)  and  the  impact  of  mission  oriented  protective 
posture  (MOPP)  gear)  can  be  evaluated  by  materiel  developers. 

4. 1.6.3  Several  of  the  HARDMAN  III  constraint  modules  (especially  M-CON  and 
P-CON)  appear  to  have  direct  applicability  to  U.S.  Army  headquarters,  field  operating 
agencies  and  support  contractors  involved  in  performing  trade-off  analyses  pertaining 
to  materiel  acquisition  activities. 

4. 1.6.4  The  R:Base  database  provides  an  integrated  customized  capability  that 
could  be  modified  with  the  most  up-to-date  RAM  data  and  fit  the  needs  of  Army  MPT 
analysts  who  evaluate  personnel  attributes  of  new  materiel  systems. 

4. 1.6.5  HARDMAN  III  is  an  excellent  tool  for  planners  to  run  mission  simulations 
involving  critical  path  analysis  to  vary  different  tasks  to  complete  a  mission. 

4.1.7  HARDMAN  III  Analytical  Limitations.  The  most  significant  limitation  in 
the  HARDMAN  III  is  the  difficulty  in  constructing  a  BCS  via  a  "cut  and  paste" 
method  using  the  HARDMAN  III  modules  and  have  the  data  flow  through  the  entire 
module.  The  MPT  analyst  should  have  the  capability  to  create  a  notional  system 
consisting  of  various  Army,  DOD,  foreign  or  industry  components  in  order  to  compute 
MPT  requirements  for  various  alternatives  and  then  do  comparisons.  HARDMAN  III 
does  not  have  the  capability  to  build  a  BCS  in  the  same  manner  as  HARDMAN  II. 
Another  drawback  of  HARDMAN  III  is  its  inability  to  accommodate  a  new  MOS  such 
as  a  combined  operator-maintainer.  With  the  high  probability  that  technological 


changes  will  occur  in  future  materiel  systems,  it  is  essential  that  HARDMAN  III 
address  the  creation  of  new  MOSs. 

4.1.8  Training  Prerequisites.  Based  on  the  surveys,  interviews  with  HARDMAN 
III  SMEs  and  attendance  at  the  three  day  training  seminar,  there  should  be  minimum 
educational  prerequisites  established  for  prospective  Army  analysts  prior  to  investing 
valuable  training  time  and  human  resources  in  the  HARDMAN  III  training  programs. 

4. 1.8.1  The  present  three  day  training  seminar  is  insufficient  by  itself  to  qualify 
the  average  U.S.  Army  MPT  analyst  as  a  confident  and  proficient  HARDMAN  III 
analyst  given  the  limited  amount  of  documentation  and  tutorial  training  available. 

4. 1.8.2  There  is  insufficient  documentation  for  a  person  unskilled  in  the 
application  of  MPT  methodologies  to  conduct  an  MPT  analysis  on  a  U.S.  Army 
materiel  system  using  the  HARDMAN  III  modules  even  if  the  modules  were  directed 
toward  typical  analysis  questions. 

4.2  CONCLUSIONS 

Based  on  the  technical  approach  used  (see  section  2),  the  following  conclusions 
can  be  made: 

4.2.1  HARDMAN  III  Modules  not  Designed  for  the  TRADOC  MPT  Analyst. 
TRADOC  DCD  analysts  are  more  concerned  with  unconstrained  operator  and 
maintainer  manpower  requirements  instead  of  artificially  constrained  or  limited 
percentages  of  available  manpower  inherent  in  the  M-CON  module.  The  interests  and 
needs  of  higher  headquarters  agencies  (e.g.,  HQ  DCSOPS,  SARDA)  are  different  from 
TRADOC  schools  (DCD,  DOTD).  For  example  the  former  are  more  concerned  with 
funding  constraints  for  budgetary  purposes  while  the  latter  organization  determines 
unrestricted  MPT  requirements.  Conversely,  TRADOC  DOTD  analysts  are  interested 
in  an  emerging  materiel  system’s  impacts  on  POIs,  instructor  manpower  requirements, 
student  man-days,  and  course  costs  -  none  of  which  are  provided  by  T-CON. 

4.2.2  Required  HARDMAN  111  Software  Enhancements.  The  enhancements  to 
the  HARDMAN  III  software  modules  (described  in  section  5  of  the  "Final  Report  for 
Concepts  on  MPT  Estimation")  are  necessary  to  rectify  the  flaws  that  currently  exist. 
This  conclusion  is  based  on  the  premise  that  other  agencies  other  than  TRADOC  may 
be  able  to  use  and/or  afford  HARDMAN  III. 

4.2.3  Cost/Benefit  Analysis  for  TRADOC.  The  HARDMAN  III  cost  analysis  is 
explained  in  detail  in  section  3.  The  cost  analysis  considered  the  following  factors: 
(1)  the  cost  to  input  new  or  updated  information;  (2)  the  workload  associated  with 
verification,  validation  and  manipulation  of  data  based  on  a  centralized  or 
decentralized  configuration;  and  (3)  initial  start-up  and  steady  state  condition  MPT 
costs.  The  tangible  and  intangible  costs  more  than  outweigh  the  advertised  benefits 
of  HARDMAN  III  and  provides  ample  justification  for  TRADOC  to  decline 
sponsorship  of  HARDMAN  III.  The  bottom  line  issue  for  TRADOC  is  that  it  can  ill 
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afford  the  initial  and  recurring  O&S  costs  associated  with  HARDMAN  III  when 
measured  against  the  anticipated  benefits  as  an  MPT  analytical  tool. 


SECTION  5.0 

RECOMMENDATIONS 


5.1  RECOMMENDATIONS 

The  recommendations  suggested  in  paragraphs  S.1.1,  5.1.4  and  S.l.S  below  are 
not  original  since  they  have  been  previously  cited  by  the  prime  contractors  who 
developed  the  HARDMAN  III  software  modules.  Several  of  these  recommendations 
were  reconfirmed  on  critiques  provided  by  students  who  have  attended  the  three  day 
training  seminars.  Recommendations  cited  in  paragraphs  5.1.2,  5.1.3  and  5.1.6  are 
based  solely  on  this  independent  HARDMAN  III  utility  assessment.  Several 
recommendations  are  also  premised  on  the  notion  that  agencies  other  than  TRADOC 
may  be  able  to  use  and/or  afford  HARDMAN  III. 

5.1.1  HARDMAN  111  Enhancements.  Proceed  with  the  recommended 
enhancements  for  HARDMAN  III  modules  as  described  is  Section  5  of  "Final  Report 
for  Concepts  on  MPT  Estimation  (Development  of  MANPRINT  Methods),"  prepared 
for  the  U.S.  Army  Research  Institute,  dated  December  1990.  Especially  important  are 
improvements  and  periodic  updates  to  the  SPARC  database.  Suggest  low  cost  fixes 
to  salvage  the  best  modules  (SPARC,  MAN-SEVAL  and  PER-SEVAL).  T-CON  should 
be  scrapped  for  the  following  reasons:  (1)  T-CON  does  not  provide  useful  information 
to  the  trainer  in  its  current  format;  (2)  T-CON  only  contains  information  on  existing 
POIs;  (3)  T-CON  fails  to  provide  the  necessary  ingredients  for  the  System  Training 
Plan  such  as  data  on  training  assumptions,  constraints  and  issues.  Project  A 
performance  data  needs  to  be  updated  to  reflect  the  current  generation  of  high  quality 
soldier  aptitudes  and  performance  capabilities. 

5.1.2  HARDMAN  Ill  Training.  It  appears  cost  effective  to  develop  a  computer 
based  training  (CBT)  tutorial  to  train  prospective  analysts  in  the  HARDMAN  III 
methodology.  A  CBT  tutorial  supplemented  by  a  demonstration  of  a  typical 
HARDMAN  III  analysis  could  replace  the  current  three-day  seminar  and  provide  a 
more  effective  training  program.  A  CBT  program  would  probably  require  an 
investment  of  $35,000  to  $45,000  to  produce.  However,  cost  savings  accruing  from 
elimination  of  the  three-day  seminar  held  twice  a  year  could  pay  for  the  CBT  within 
two  to  three  years  (assumes  $10,000  per  training  seminar  for  travel,  per  diem,  and 
salaries  conducted  by  two  out-of-town  contractors,  two  ARL  monitors,  and  20 
government  students  [half  from  out-of-townl).  Use  of  CBT  would  also  facilitate 
version  updates  and  make  configuration  control  much  easier.  Another  possible  frugal 
alternative  is  to  produce  a  video  cassette  recording  (VCR)  to  provide  basic 
familiarization  training  to  those  individuals  requesting  an  overview  of  the  HARDMAN 
III  modules  who  will  not  be  performing  actual  MPT  analyses  on  materiel  systems 
(e.g.,  OSD  officials,  other  services,  contractors,  etc.). 
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5.1.3  Develop  Personnel  Selection  Criteria.  In  order  to  get  the  best  possible 
return  on  the  investment  in  training  of  new  HARDMAN  III  analysts,  there  needs  to 
be  personnel  selection  criteria  for  both  military  and  Department  of  the  Army  civilian 
analysts  for  selection  as  HARDMAN  III  analysts.  Criteria  should  be  in  the  form  of 
prerequisite  skills,  knowledge  and  abilities  (SKAs).  Note:  These  criteria  should  be 
used  by  the  Army  as  a  guide  only  and  are  not  intended  to  change  civilian  personnel 
regulations.  For  example,  a  typical  HARDMAN  III  analyst  might  possess  the 
following  qualifications: 

S.  1.3.1  Bachelor’s  degree  in  a  technical,  training,  or  educational  field  of  study. 
Knowledge  of  operations  research/systems  analysis  (ORSA),  or  statistical  modeling  is 
desired. 

5. 1.3.2  Minimum  of  two  years  computer  literacy  in  the  Micro  Soft  Disk  Operating 
System  (MS-DOS). 

5. 1.3.3  At  least  three  years  experience  associated  with  a  U.S.  Army  mission  area 
or  materiel  acquisition  process  to  which  the  analyst  will  subsequently  be  assigned. 

5. 1.3.4  Minimum  of  two  years  experience  in  the  application  of  MPT 
methodologies  and  techniques  like  HARDMAN,  Training  Impact  Analysis  t^TIA),  and 
other  MANPRINT  analyses. 

5.1.4  Consolidate  Disparate  Databases.  Consolidating  the  various  databases  into 
a  single  library  and  possibly  consolidating  several  of  the  modules  together  would 
rectify  many  of  the  inconsistencies  inherent  in  HARDMAN  III.  The  following 
benefits  could  be  realized: 

5. 1.4.1  A  consolidated  database  would  be  easier  to  control  and  verify.  Also,  it 
would  be  much  easier  to  keep  one  database  updated  than  the  current  six. 

5. 1.4.2  Along  with  the  library  database,  all  work  files  should  be  consolidated  as 
well  to  eliminate  input  redundancies. 

S.  1.4.3  This  would  reduce  user  workload  and  data  file  inconsistencies  among 
modules  as  well. 

5. 1.4.4  By  consolidating  several  of  the  modules,  workload  could  be  reduced  as 
well  as  disk  space  since  many  of  the  inherent  redundancies  would  be  eliminated. 

5.1.5  Improve  Module  Linkages.  Recommend  improving  the  linkages  among 
modules  to  facilitate  the  transfer  of  data  since  data  from  one  module  is  similar  to  data 
required  by  another  module.  Currently,  transfer  of  data  is  not  possible  in  all  cases, 
e.g.,  data  in  P-CON  cannot  be  transferred  to  PER-SEVAL,  although  both  deal  with 


personnel  issues  (Note:  this  recommendation  is  based  on  the  premise  that  there  is 
some  HARDMAN  III  utility  to  Army  agencies  outside  of  TRADOC). 

S.1.6  TRADOC  Sponsorship.  Even  with  the  enhancements  to  salvage  the 
HARDMAN  III  software  modules,  we  do  not  recommend  the  sponsorship  of  the 
HARDMAN  III  software  by  Headquarters  TRADOC  for  the  reasons  cited  in  Section  4. 
TRADOC  school  Combat  Development  Directorates  analysts  require  support  in 
developing,  defining  and  evaluating  a  real  or  conceptual  materiel  system  for  input  into 
the  Operational  Requirements  Document  (ORD).  However,  the  HARDMAN  III 
modules  were  not  designed  with  DCD  user  in  mind  for  the  reasons  previously  cited 
in  section  4.  In  the  event  TRADOC  schools  require  additional  analytical  support, 
recommend  contractor  augmentation.  For  example,  the  U.S.  Army  Air  Defense 
Artillery  School  and  Center  (USAADASCH)  at  Fort  Bliss  and  the  U.S.  Army 
Intelligence  Center  and  School  (USAICS)  at  Fort  Huachuca  are  programmed  to  receive 
a  number  of  additional  materiel  systems  between  now  and  the  year  2000.  It  may  be 
prudent  for  these  schools  to  contract  out  for  the  application  of  HARDMAN  III  since 
the  O&S  costs  associated  with  HARDMAN  III  appear  to  outweigh  the  potential 
benefits  derived  from  determining  new  materiel  systems  MPT  requirements,  concerns 
and  issues  assuming  ultimate  verification  and  validation. 

In  summary,  ARL’s  Human  Research  and  Engineering  Directorate  has  met  its 
primary  organizational  goal  and  charter  and  has  produced  an  MPT  analytical  tool  for 
possible  use  by  Army  organizations,  DoD  agencies  and  prime  contractors.  However, 
the  utility  of  the  HARDMAN  III  software  is  suspect  until  appropriate  software 
modifications  are  completed  (including  verification  and  validation),  documentation  is 
updated  to  reflect  current  software  configurations,  and  an  adequate  training  package 
is  developed  to  provide  potential  users  with  proper  instruction  regarding  the 
applications,  functions,  and  operations  of  the  HARDMAN  III  software. 
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ACRONYMS 


AFAS .  Advanced  Field  Artillery  System 

AGS  .  Armored  Gun  System 

AIR .  American  Institutes  for  Research 

AMC .  Army  Materiel  Command 

ARI .  Army  Research  Institute 

ARL  .  Army  Research  Laboratory 

ASA  .  Applied  Science  Associates,  Inc. 

ASARC .  The  Army  Systems  Acquisition  Review  Council 

ASVAB .  Armed  Services  Vocational  Aptitude  Battery 

ATRRS .  Army  Training  Requirements  and  Resources  System 

BCE  .  Baseline  Cost  Estimate 

BCS .  Baseline  Comparison  System 

CARD  .  Cost  Analysis  Requirements  Document 

CBT  .  Computer  Based  Training 

CBRS .  Concept  Based  Requirements  System 

COEA  .  Cost  and  Operational  Effectiveness  Analyses 

DCD  .  Directorate  of  Combat  Development 

DCSPI  .  Deputy  Chief  of  Staff  for  Personnel  Integration 

DEC  .  Digital  Equipment  Corporation 

DOD  .  Department  of  Defense 

DOTD  .  Directorate  of  Training  Development 

DRC  .  Dynamics  Research  Corporation 

DTIC .  Defense  Technical  Information  Center 

ECA  .  Early  Comparative  Analysis 

EGA  .  Enhanced  Graphics  Adapter 

FARP .  Forward  Area  Refueling  Point 

FORCE .  Force  Analysis  Aid 

FY  .  Fiscal  Year 

HARDMAN  ....  Hardware  versus  Manpower 

HCM .  HARDMAN  Comparative  Methodology 

HOS  .  Human  Operator  Simulator 
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HEL  .  . 
HumRRO 


Human  Engineering  Laboratory 
Human  Resources  Research  Organization 


MAC .  Macintosh  Computer 

MAD .  Micro  Analysis  and  Design,  Inc. 

MAISRC .  Major  Automated  Information  Systems  Review  Council 

MANCAP  .  Manpower  Capability  Aid 

MANPRINT  ....  Manpower  and  Personnel  Integration 
MAN-SEVAL  .  .  .  Manpower-Based  System  Evaluation  Aid 

MARC  .  Manpower  Requirements  Criteria 

M-CON .  Manpower  Constraints  Aid 

MER .  Manpower  Estimate  Report 

MIL-STD .  Military  Standard 

MIST .  Man  Integrated  Systems  Technology 

MMI  .  Man-Machine  Interface 

MOPP  .  Mission  Oriented  Protective  Posture 

MOS  .  Military  Occupational  Specialty 

MPT  .  Manpower,  Personnel  and  Training 

MS-DOS  .  Micro  Soft  Disk  Operating  System 

NAVES .  Navigation  Emplacement  System 

NDI .  Nondevelopment  item 

O&S  .  Operation  and  Support 

ORD  .  Operational  Requirements  Document 

ORSA .  Operations  Research/Systems  Analysis 

OSD  .  Office  of  the  Secretary  of  Defense 

OTEA .  Operational  Test  and  Evaluation  Agency 

PC  .  Personal  Computer 

P-CON .  Personnel  Constraints  Aid 

PDRI .  Personnel  Decisions  Research  Institute 

PEO  .  Program  Executive  Officer 

PERSCOM .  U.S.  Total  Army  Personnel  Command 

PER-SEVAL  ....  Personnel-Based  System  Evaluation  Aid 
PM .  Program  Manager 

PNAVES .  Patriot  Navigation  Emplacement  System 

POIs  .  Programs  of  Instruction 

QEMM .  Quarterdeck  Extended  Memory  Manager 
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RAM .  Random  Access  Memory 

RAM .  Reliability,  Availability  and  Maintainability 

RBF .  RiBase  File 

SARDA .  Secretary  of  the  Army  for  Research,  Development  and  Acquisition 

SAT .  Systems  Approach  to  Training 

SKA  .  Skills,  Knowledge,  and  Ability 

SMEs .  Subject  Matter  Experts 

SMMP  .  System  MANPRINT  Management  Plan 

SPARC .  System  Performance  and  RAM  Criterion  Estimation  Aid 

SQT .  Skills  and  Qualification  Tests 

T-CON .  Training  Constraints  Estimation  Aid 

TIA .  Training  Impact  Analysis 

TRAC .  TRADOC  Analysis  Command 

TRADOC .  Training  and  Doctrine  Command 

USAADASCH  ...  U.S.  Army  Air  Defense  Artillery  School  and  Center 

USAICS  .  U.S.  Army  Intelligence  Center  and  School 

USTAPC .  U.S.  Total  Army  Personnel  Center 

VCR  .  Video  Cassette  Recording 

VGA  .  Video  Graphics  Adapter 
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LIST  OF  DOCUMENTS 


The  following  list  of  referenced  documents,  reports,  software  and  other  pertinent 
information  has  been  reviewed  as  part  of  the  HARDMAN  III  utility  assessment 
including  a  Manpower,  Personnel  and  Training  (MPT)  analysis: 

•  Final  Report  (E-17611U)  for  Concepts  on  MPT  Estimation  (Development  of 
MANPRINT  Methods)  produced  by  Dynamic  Research  Corporation,  Inc.  and  Micro 
Analysis  &  Design,  December  1990  for  the  U.S.  Army  Research  Institute  (ARI) 

•  Armored  Gun  System  (AGS)  MPT  data  on  computer  disks 

•  Software  and  User  Guides  for  all  six  HARDMAN  modules: 

System  Performance  and  RAM  Criterion  Aid  (SPARC) 

—  Manpower  Constraints  Aid  (M-CON) 

—  Personnel  Constraints  Aid  (P-CON) 

—  Training  Constraints  Aid  (T-CON) 

—  Manpower-based  System  Evaluation  Aid  (MAN-SEVAL) 

—  Personnel-based  System  Evaluation  Aid  (PER-SEVAL) 

•  HARDMAN  III  Training  Course  (November  7-9,  1990  and  29  June-1  July  1992) 
handouts  developed  by  Micro  Analysis  &  Design  Inc.  and  Dynamics  Research  Corp 

•  Walk-Through  of  HARDMAN  III  briefing  slides  given  to  U.S.  Army  TRADOC,  22- 
23  April  1992,  by  Drs.  Laurel  Allender  and  Engin  Crosby  of  the  U.S.  Army 
Research  Laboratory 

•  AGS  Cost  and  Operational  Effectiveness  Analysis  (COEA)  Manpower  and  Personnel 
Analysis  Plan,  22  March  1991 

•  HARDMAN  III  briefing  given  to  Col  Macey  HQ  TRADOC  by  LTC  Correia  on  13 
February  1992 

•  HARDMAN  II  Comparability  Methodology,  Seven  Volume  Report,  May  1990, 
developed  for  the  Manned  Systems  Group  (SRL),  U.S.  Army  Research  Institute 

•  HARDMAN,  Five  Volume  Report,  April  1985,  prepared  for  the  Technical 
Information  Division,  U.S.  Army  Research  Institute 


•  Man  Integrated  Systems  Technology  (MIST)  Users  Guide  produced  by  Dynamics 
Research  Corp.,  4  September  1985,  for  the  U.S.  Army  Research  Institute 

•  DoD  Directive  7920.1,  ”Life  Cycle  Management  of  Automated  Information 
Systems,"  June  20,  1988 

•  DoD-STD-2167A,  "Defense  System  Software  Development" 

•  DoD-STD-2168,  "Defense  System  Software  Quality  Program" 

•  Development  and  Field  Test  of  the  Trial  Battery  for  Project  A,  May  1987,  produced 
by  the  U.S.  Army  Research  Institute 

•  Draft  Advanced  Field  Artillery  System  (AFAS)  Task  List,  July  31  1992,  produced 
for  the  U.S.  Army  Research  Institute,  Fort  Sill  Field  Unit  by  CAE-Link 
Corporation  of  Falls  Church,  VA  using  HARDMAN  III 
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PHASE  0  -  CONCEPT  EXPLORATION  &  DERNITION  PHASE 
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PHASE  III  -  DEPLOYMENT  PHASE 

MILESTONE  III 
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HARDMAN  III  User  Survey 

Date:  _ 

Organization: _ 

Location: _ 

Job  series/Specialty/MOS: _ 

Position  Title:  _ 

Military  Education:  _ 

Civilian  Education:  _ 

1 .  Describe  your  current  duties  and  responsibilities: 

Include  the  percentage  of  time  you  spend  on  analysis. 

2 .  Describe  your  MANPRINT/MPT/COEA  analysis  and  study  experience: 

3 .  If  you  attended  the  HARDMAN  III  Training  seminar,  how  well  did  it: 

a.  help  you  understand  the  HARDMAN  III  methodology? 

b.  prepare  you  to  conduct  analysis  with  the  six  modules? 

c.  help  you  to  understand  what  each  module  does? 


4.  To  your  knowledge,  does  each  of  the  six  HARDMAN  III  modules  have  a  tutorial?  If  yes, 
how  effective  are  they  in  teaching  the  user  the  proper  steps? 
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5.  If  you  conduct  MANPRINT/MPT/COEA  analyses,  describe  your  confidence  in  your 
abilities  to  conduct  analyses  using  HARDMAN  III? 


6.  What  costs  (money,  time,  personnel,  other  resources)  do  you  feel  are  associated  with  your 
current  and  potential  use  of  HARDMAN  III? 


7 .  What  benefits  (savings  of  resources)  do  you  feel  are  (or  could  be)  associated  with  your 
current  (and  potential)  use  of  HARDMAN  III? 


8 .  Have  you  used  HARDMAN  HI  to  analyze  any  actual  or  “notional”  Army  materiel  system? 

If  yes,  describe  your  analysis  including  modules  used  and  systems  analyzed. 


9.  Have  you  verified  any  HARDMAN  III  algorithms  by  manipulating  any  actual  MPT  data? 
If  yes,  describe  your  analysis  including  modules  us^,  data  analyzed,  and  the  source  of 
your  data. 


1 0.  Please  comment  on  your  perception  of  the  library  systems  and  data  built  into  HARDMAN 
III. 


1 1 .  What  iTKxlifications  and  enhancements  would  you  recommend  for  the  six  software 
modules? 


12.  What  prerequisites  would  you  recommend  for  attendance  at  the  training  seminar  (job 
requirements,  PC  knowledge)? 


1 3 .  What  is  your  assessment  of  the  overall  utility  of  HARDMAN  III? 
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14.  What  other  comments  or  recommendations  do  you  have  to  improve  HARDMAN  III?  Use 
additional  sheets  as  necessary. 


Thank  you  for  your  assistance. 
Optional  Name: _ 
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Phone: 
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